Methods and apparatus for memory allocation and reallocation in networking stack infrastructures

ABSTRACT

Methods and apparatus for memory allocation and reallocation in networking stack infrastructures. Unlike prior art monolithic networking stacks, the exemplary networking stack architecture described hereinafter includes various components that span multiple domains (both in-kernel, and non-kernel). For example, unlike traditional “socket” based communication, disclosed embodiments can transfer data directly between the kernel and user space domains. A user space networking stack is disclosed that enables extensible, cross-platform-capable, user space control of the networking protocol stack functionality. The user space networking stack facilitates tighter integration between the protocol layers (including TLS) and the application or daemon. Exemplary systems can support multiple networking protocol stack instances (including an in-kernel traditional network stack). Due to this disclosed architecture, physical memory allocations (and deallocations) may be more flexibly implemented.

PRIORITY

This application claims the benefit of priority to U.S. ProvisionalPatent Application Ser. No. 62/649,509 filed Mar. 28, 2018 and entitled“METHODS AND APPARATUS FOR EFFICIENT DATA TRANSFER WITHIN USER SPACENETWORKING STACK INFRASTRUCTURES”, which is incorporated herein byreference in its entirety.

RELATED APPLICATIONS

This application is related to U.S. patent application Ser. No.16/144,992 filed Sep. 27, 2018 and entitled “Methods and Apparatus forSingle Entity Buffer Pool Management”, U.S. patent application Ser. No.16/146,533 filed Sep. 28, 2018 and entitled “Methods and Apparatus forRegulating Networking Traffic in Bursty System Conditions”, U.S. patentapplication Ser. No. 16/146,324 filed Sep. 28, 2018 and entitled“Methods and Apparatus for Preventing Packet Spoofing with User SpaceCommunication Stacks”, U.S. patent application Ser. No. 16/146,916 filedSep. 28, 2018 and entitled “Methods and Apparatus for Channel DefunctWithin User Space Stack Architectures”, U.S. patent application Ser. No.16/236,032 filed Dec. 28, 2018 and entitled “Methods and Apparatus forClassification of Flow Metadata with User Space Communication Stacks”,U.S. patent application Ser. No. 16/363,495 filed Mar. 25, 2019 andentitled “Methods and Apparatus for Dynamic Packet Pool Configuration inNetworking Stack Infrastructures”, U.S. patent application Ser. No.16/365,462 filed Mar. 26, 2019 and entitled “Methods and Apparatus forSharing and Arbitration of Host Stack Information with User SpaceCommunication Stacks”, U.S. patent application Ser. No. 16/365,484 filedMar. 26, 2019 and entitled “Methods and Apparatus for VirtualizedHardware Optimizations for User Space Networking”, U.S. patentapplication Ser. No. ______, filed concurrently herewith on Mar. 28,2019 and entitled “Methods and Apparatus for Secure Operation of UserSpace Communication Stacks”, U.S. patent application Ser. No. ______,filed concurrently herewith on Mar. 28, 2019 and entitled “Methods andApparatus for Active Queue Management in User Space Networking”, andU.S. patent application Ser. No. ______, filed concurrently herewith onMar. 28, 2019 and entitled “Methods and Apparatus for Self-TuningOperation within User Space Stack Architectures”, each of the foregoingbeing incorporated herein by reference in its entirety.

COPYRIGHT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

1. TECHNICAL FIELD

The disclosure relates generally to the field of electronic devices, aswell as networks thereof. More particularly, the disclosure is directedto methods and apparatus for implementing computerized networking stackinfrastructures. Various aspects of the present disclosure are directedto, in one exemplary aspect, data transfer within user space networkingstack infrastructures.

2. DESCRIPTION OF RELATED TECHNOLOGY

The consumer electronics industry has seen explosive growth in networkconnectivity; for example, Internet connectivity is now virtuallyubiquitous across many different device types for a variety of differentapplications and functionalities. The successful implementation ofnetwork connectivity over a myriad of different usage cases has beenenabled by, inter alia, the principles of modular design andabstraction. Specifically, the traditional network communicationparadigm incorporates multiple (generally) modular software “layers”into a “communication stack.” Each layer of the communication stackseparately manages its own implementation specific considerations, andprovides an “abstracted” communication interface to the next layer. Inthis manner, different applications can communicate freely acrossdifferent devices without considering the underlying network transport.

The traditional network communication paradigm has been relativelystable for over 30 years. The Assignee hereof has developed its ownimplementation of a computer networking stack (based on the traditionalnetworking paradigm) that is mature, robust, and feature-rich (yetconservative). This networking stack is the foundation for virtually allnetworking capabilities, including those used across the Assignee'sproducts (e.g., MacBook®, iMac®, iPad®, and iPhone®, etc.) and has beendesigned to handle a variety of protocols (such as TCP (TransmissionControl Protocol), UDP (User Datagram Protocol) and IP (InternetProtocol)), and proprietary extensions and functionalities.

While the traditional network communication paradigm has many benefits,changes in the commercial landscape have stretched the capabilities ofthe existing implementations. Over the past years new use cases haveemerged that require capabilities beyond those of the traditionalnetworking stack design. For example, some use cases require control anddata movement operations to be performed in so-called “user space”(software that is executed outside the kernel, and specific to a userprocess). Common examples of such applications include withoutlimitation e.g. Virtual Private Networks (VPN), application proxy,content and traffic filtering, and any number of other network-awareuser applications.

Furthermore, certain types of user applications (e.g., media playback,real-time or interactive network applications) would benefit fromworkload-specific customizations and performance optimizations of thenetworking stack.

Unfortunately, the current one-size-fits-all networking stack was notdesigned for (and is thus ill-suited to) the requirements of theaforementioned use cases (and others contemplated herein). Moredirectly, supporting user space applications and associated componentsfrom within the traditional in-kernel networking stack architecture addscomplexity, increases technical debts (the implied cost of reworkattributed to deploying a faster, but suboptimal, implementation),brings in higher processing costs, and results in suboptimal performanceand higher power consumption.

To these ends, a networking stack architecture and technology thatcaters to emerging non-kernel use cases is needed. Ideally, but not as arequisite, such solutions should preserve backwards compatibility withthe traditional in-kernel networking stack. More generally, improvedmethods and apparatus for manipulating and/or controlling lower layernetworking communication protocols by higher layer software applicationsis desired.

SUMMARY

The present disclosure satisfies the foregoing needs by providing, interalia, methods and apparatus for data transfer within user spacenetworking stack infrastructures.

In one aspect, a system for managing pools of resources is disclosed. Inone embodiment, the system includes one or more processor apparatus; oneor more applications executable by the one or more processor apparatus;physical memory for use by the one or more applications; and aninput/output memory management unit (IOMMU) for use by the one or moreapplications, the IOMMU configured to access a kernel virtual address(KVA), the KVA providing for translation of a virtual address accessedby the one or more applications to a physical address associated withthe physical memory. The one or more processor apparatus are configuredto: allocate a virtual address space to one of the one or moreapplications; associate the virtual address space with a portion of thephysical memory via use of the IOMMU; and de-allocate a subset of theportion of the physical memory associated with the virtual addressspace.

In one variant, the virtual address space includes an arena, the arenaincludes a plurality of regions, each of the plurality of regionsincludes a plurality of segments, each of the segments includes aplurality of objects.

In another variant, the deallocated subset of the portion of thephysical memory is associated with a first segment of the plurality ofsegments.

In yet another variant, the deallocation occurs in accordance with adeferred context, and the deferred context includes an expiration of atimer.

In yet another variant, the deallocation occurs in accordance with adeferred context, and the deferred context includes a context switchbetween user space and kernel space.

In yet another variant, the deallocation of the subset of the portion ofthe physical memory occurs responsive to signaling from a user processto a kernel process.

In yet another variant, the signaling from the user process to thekernel process is indicative that the one of the one or moreapplications is in an active state, the active state associated with anactive threshold value.

In yet another variant, the signaling from the user process to thekernel process is indicative that the one of the one or moreapplications is in an inactive state, the inactive state associated withan inactive threshold value, the inactive threshold value being lowerthan the active threshold value.

In yet another variant, the one or more processor apparatus are furtherconfigured to place the one of the one or more applications from aforeground process to a background process; receive control plane packetdata, the received control plane packet data including control planeheader data and control plane payload data; replace the control planepayload data with null data; and copy the null data and the controlplane header data into heap memory.

In yet another variant, the one or more processor apparatus are furtherconfigured to read the null data and the control plane header data inthe heap memory; and resume network connectivity for the one of the oneor more applications.

In another aspect, methods and apparatus for requesting a channel schemafor an application is disclosed. In one embodiment, the method includesrequesting the channel schema for the application; and assigning aplurality of data structures associated with the requested channelschema, a portion of the plurality of data structures being configuredin accordance with application requirements for the application.

In one variant, the assigning of the plurality of data structuresincludes assigning a transmission buffer having a first buffer size; andassigning a receive buffer having a second buffer size.

In another variant, the method further includes receiving a request fora different size for one or both of the first buffer size and the secondbuffer size.

In yet another variant, the receiving of the request for the differentsize for one or both of the first buffer size and the second buffer sizeincludes increasing a data throughput for one or both of transmissionbuffer and the receive buffer.

In yet another variant, the receiving of the request for the differentsize for one or both of the first buffer size and the second buffer sizeincludes decreasing an amount of latency for one or both of transmissionbuffer and the receive buffer.

In yet another variant, the method further includes mirroring theplurality of data structures between a user-accessible region and akernel-only accessible region; and applying a common offset from a baseaddress within the user-accessible region and the kernel-only accessibleregion for a metadata user object and a metadata kernel objectassociated with the metadata user object.

In yet another aspect, methods and apparatus for deallocating one ormore physical memory pages from a kernel virtual address space isdisclosed. In one embodiment, the method includes assigning the kernelvirtual address space to an application; backing the kernel virtualaddress space with physical memory; determining that a portion of thekernel virtual address space is underutilized; and deallocating theportion of the kernel virtual address space that is underutilized fromthe physical memory.

In one variant, the deallocating of the portion of the kernel virtualaddress space that is underutilized from the physical memory includescompressing contents from the physical memory that is associated withthe portion of the kernel virtual address space that is underutilized;and placing the compressed contents from the physical memory into anon-volatile memory storage device.

In another variant, the method further includes accessing the portion ofthe virtual address space that has been deallocated, the accessingincluding decompressing the compressed contents in the non-volatilememory storage device; and placing the decompressed contents in thenon-volatile memory storage device into the physical memory.

In yet another variant, the placing of the decompressed contents in thenon-volatile memory storage device into the physical memory includesplacing the decompressed contents into another area of the physicalmemory, while the accessing of the portion of the virtual address spacecomprises using a same virtual address space addressing for the contentsfrom the physical memory.

In yet another aspect, methods and apparatus for user pipe dynamicmemory management using sync statistics are disclosed. In oneembodiment, a user pipe process (e.g., Nexus) provides an efficientinter-processor communication (IPC) between user space processes usingshared memory. Since the number of processes using IPC on an iOS devicecan be significant, an efficient mechanism is provided so as to keep theshared memory usage to minimum without compromising on the datathroughput. In one variant, maintaining a fair estimate of immediatememory usage of user (working set) depending on the recent past usage isperformed; the user pipe Nexus maintains a weighted moving averagestatistics of memory used during each synchronization, and can also keepadjusting the channel memory accordingly as needed.

In yet another aspect, methods and apparatus for purgeable memory(compressible and swappable) are disclosed. In one embodiment, the USNSIarchitecture allocates all memory as purgeable and wires memory ondemand when needed.

In yet another aspect, methods and apparatus for memory region/arena:purpose, layout, access protection, sharing model are disclosed. In oneembodiment, an efficient and generic mechanism to represent and managethe shared memory objects of varying types and sizes which are memorymapped to the user space and/or kernel space is disclosed. In one suchembodiment, the USNSI architecture uses shared memory for efficientpacket I/O, network statistics and system attributes (sysctl). USNSIarena is a generic and efficient mechanism to represent these varioustypes of shared memory subsystems and their backing memory caches,regions and access protection attributes. Channel schema is arepresentation of the shared memory layout for user space process to beable to efficiently access various channel objects.

In yet another aspect, methods and apparatus for mirrored memory regionsare disclosed. In one embodiment, to implement security validation andsanitation of shared memory objects on user-kernel boundary, akernel-only copy of these objects is maintained, and an efficient methodto allocate and retrieve these objects is provided. In one variant,mirrored memory object(s) is/are created, which share the same regionoffset as that of the associated object and hence can be retrievedquickly from the attributes of the associated object.

In yet another aspect, methods and apparatus for channeling defunct (mapoverrides) are disclosed. In one embodiment, networking memoryassociated with a process when it is backgrounded is freed; redirectionof the shared memory mapping of the task so that they are backed withanonymous (zero-filled) pages is used to free the underlying memory.When the task is resumed, the user space shared memory accessorfunctions (e.g., libsyscall wrappers) have the logic to detect adefuncted state of the shared memory, and efficiently and effectivelyhandle errors due to data inconsistencies.

In yet another aspect, methods and apparatus for conducting one or more“reaps” based on idleness are disclosed. In one embodiment, efficientand aggressive pruning and purging of idle resources are utilized via,inter alia, mechanisms which can detect idle resources and can offloadpruning and purging of these resources in a deferred context.

In yet another aspect, methods and apparatus for management of daemon“jetsam” are disclosed. In one embodiment, a memory management modulethat keeps track of the memory consumed by the network protocols isprovided; depending on memory usage, the module indicates to the systemthat active work is being performed by the protocols on behalf of theapplication. Once the buffers are returned to the memory managementmodule, the module indicates to the system that the active work iscomplete. This prevents the system from targeting processes that consumemore memory while doing active work.

In yet another aspect, methods and apparatus for TCP memory “defunct”management are disclosed. In one embodiment, data inconsistency issueswhen a channel is defunct during processing of TCP packet are avoided byuse of a shadow copy of the original TCP header in heap memory. Once TCPprocessing begins, it uses the copy of the TCP header to make decisionswhich prevents any inconsistency or data corruption. The validation isdone prior to handing off the payload data to the layer above TCP, aswell as within the TCP input processing paths.

In yet another aspect, a computerized device implementing one or more ofthe foregoing aspects is disclosed and described. In one embodiment, thedevice includes a personal or laptop computer. In another embodiment,the device includes a mobile device (e.g., tablet or smartphone).

In yet another aspect, an integrated circuit (IC) device implementingone or more of the foregoing aspects is disclosed and described. In oneembodiment, the IC device is embodied as a SoC (system on Chip) device.In another embodiment, an ASIC (application specific IC) is used as thebasis of the device. In yet another embodiment, a chip set (i.e.,multiple ICs used in coordinated fashion) is disclosed.

In yet another aspect, a computer readable storage apparatusimplementing one or more of the foregoing aspects is disclosed anddescribed. In one embodiment, the computer readable apparatus comprisesa program memory, or an EEPROM. In another embodiment, the apparatusincludes a solid state drive (SSD) or other mass storage device. Inanother embodiment, the apparatus comprises a USB or other “flash drive”or other such portable removable storage device. In yet anotherembodiment, the apparatus comprises a “cloud” (network) based storagedevice which is remote from yet accessible via a computerized user orclient electronic device.

In yet another aspect, a software architecture is disclosed. In oneembodiment, the architecture includes both user space and kernel space,separated via a software or virtual partition.

Other features and advantages of the present disclosure will immediatelybe recognized by persons of ordinary skill in the art with reference tothe attached drawings and detailed description of exemplary embodimentsas given below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a logical representation of a traditional network socket,useful for explaining various aspects of the present disclosure.

FIG. 2 is a logical representation of a computer system that implementsInput/Output (I/O) network control, useful for explaining variousaspects of the present disclosure.

FIG. 3 is a logical block diagram of one exemplary implementation ofTransport Layer Security (TLS), useful for explaining various aspects ofthe present disclosure.

FIG. 4 is a logical block diagram of an exemplary implementation of aVirtual Private Network (VPN), useful for explaining various aspects ofthe present disclosure.

FIG. 5 is a logical block diagram of an exemplary implementation ofapplication based tuning, useful for explaining various aspects of thepresent disclosure.

FIG. 6 is a logical representation of an exemplary networking stackarchitecture, in accordance with the various aspects of the presentdisclosure.

FIG. 7 is a logical block diagram of an exemplary user space networkingstack, in accordance with the various aspects of the present disclosure.

FIG. 8 is a logical flow diagram useful to summarize the convoluted datapath taken for a prior art application using a proxy agent applicationwithin the context of the traditional networking stack, useful forexplaining various aspects of the present disclosure.

FIG. 9 is a logical flow diagram useful to summarize an exemplary proxyagent application within the context of the user space networking stack,in accordance with various aspects of the present disclosure.

FIG. 10A is a logical block diagram illustrating problematic operationfor user space communication stacks, in accordance with various aspectsof the present disclosure.

FIG. 10B is a logical block diagram illustrating an exemplary solutionto the problematic operation of user space communication stacks, inaccordance with various aspects of the present disclosure.

FIG. 11 is a logical block representation illustrating an exemplaryoperating scenario that has been generalized to any sort of controlplane data, in accordance with various aspects of the presentdisclosure.

FIG. 12 is a logical block diagram of an exemplary implementation of apacket pool for a non-kernel space application, in accordance withvarious aspects of the present disclosure.

FIG. 13 is a logical representation illustrating an exemplary kernelvirtual address allocation, in accordance with various aspects of thepresent disclosure.

FIG. 14 is a logical block diagram of an exemplary system forimplementing the various methodologies described herein, in accordancewith various aspects of the present disclosure.

FIG. 15 is a logical representation of mirrored memory allocations, inaccordance with various aspects of the present disclosure.

FIG. 16 is a generalized method for the allocation/deallocation ofphysical memory, in accordance with various aspects of the presentdisclosure.

All figures © Copyright 2017-2019 Apple Inc. All rights reserved.

DETAILED DESCRIPTION

Reference is now made to the drawings, wherein like numerals refer tolike parts throughout.

Detailed Description of Exemplary Embodiments

Exemplary embodiments of the present disclosure are now described indetail. While embodiments are primarily discussed in the context of usein conjunction with an inter-processor communication (IPC) link such asthat described in, for example, commonly owned U.S. patent applicationSer. No. 14/879,024 filed Oct. 8, 2015 and entitled “METHODS ANDAPPARATUS FOR RUNNING AND BOOTING AN INTER-PROCESSOR COMMUNICATION LINKBETWEEN INDEPENDENTLY OPERABLE PROCESSORS”, now U.S. Pat. No.10,078,361, and co-owned and co-pending U.S. patent application Ser. No.16/112,480 filed Aug. 24, 2018 and entitled “METHODS AND APPARATUS FORCONTROL OF A JOINTLY SHARED MEMORY-MAPPED REGION”, each of which beingincorporated herein by reference in its entirety, it will be recognizedby those of ordinary skill that the present disclosure is not solimited.

Existing Network Socket Technologies—

FIG. 1 illustrates one logical representation of a traditional networksocket 102, useful for explaining various aspects of the traditionalnetworking interface. A network “socket” is a virtualized internalnetwork endpoint for sending or receiving data at a single node in acomputer network. A network socket may be created (“opened”) ordestroyed (“closed”) and the manifest of network sockets may be storedas entries in a network resource table which may additionally includereference to various communication protocols (e.g., Transmission ControlProtocol (TCP) 104, User Datagram Protocol (UDP) 106, Inter-ProcessorCommunication (IPC) 108, etc.), destination, status, and any otheroperational processes (kernel extensions 112) and/or parameters); moregenerally, network sockets are a form of system resource.

As shown in FIG. 1, the socket 102 provides an application programminginterface (API) that spans between the user space and the kernel space.An API is a set of clearly defined methods of communication betweenvarious software components. An API specification commonly includes,without limitation: routines, data structures, object classes,variables, remote calls and/or any number of other software constructscommonly defined within the computing arts.

As a brief aside, user space is a portion of system memory that aprocessor executes user processes from. User space is relatively freelyand dynamically allocated for application software and a few devicedrivers. The kernel space is a portion of memory that a processorexecutes the kernel from. Kernel space is strictly reserved (usuallyduring the processor boot sequence) for running privileged operatingsystem (O/S) processes, extensions, and most device drivers. Forexample, each user space process normally runs in a specific memoryspace (its own “sandbox”), and cannot access the memory of otherprocesses unless explicitly allowed. In contrast, the kernel is the coreof a computer's operating system; the kernel can exert complete controlover all other processes in the system.

The term “operating system” may refer to software that controls andmanages access to hardware. An 0/S commonly supports processingfunctions such as e.g., task scheduling, application execution, inputand output management, memory management, security, and peripheralaccess. As used herein, the term “application” refers to software thatcan interact with the hardware only via procedures and interfacesoffered by the O/S.

The term “privilege” may refer to any access restriction or permissionwhich restricts or permits processor execution. System privileges arecommonly used within the computing arts to, inter alia, mitigate thepotential damage of a computer security vulnerability. For instance, aproperly privileged computer system will prevent malicious softwareapplications from affecting data and task execution associated withother applications and the kernel.

As used herein, the term “in-kernel” and/or “kernel space” may refer todata and/or processes that are stored in, and/or have privilege toaccess to, the kernel space memory allocations. In contrast, the terms“non-kernel” and/or “user space” refers to data and/or processes thatare not privileged to access the kernel space memory allocations. Inparticular, user space represents the address space specific to the userprocess, whereas non-kernel space represents address space which is notin-kernel, but which may or may not be specific to user processes.

As previously noted, the illustrated socket 102 provides access toTransmission Control Protocol (TCP) 104, User Datagram Protocol (UDP)106, and Inter-Processor Communication (IPC) 108. TCP, UDP, and IPC arevarious suites of transmission protocols each offering differentcapabilities and/or functionalities. For example, UDP is a minimalmessage-oriented encapsulation protocol that provides no guarantees tothe upper layer protocol for message delivery and the UDP layer retainsno state of UDP messages once sent. UDP is commonly used for real-time,interactive applications (e.g., video chat, voice over IP (VoIP)) whereloss of packets is acceptable. In contrast, TCP provides reliable,ordered, and error-checked delivery of data via a retransmission andacknowledgement scheme; TCP is generally used for file transfers wherepacket loss is unacceptable, and transmission latency is flexible.

As used herein, the term “encapsulation protocol” may refer to modularcommunication protocols in which logically separate functions in thenetwork are abstracted from their underlying structures by inclusion orinformation hiding within higher level objects. For example, in oneexemplary embodiment, UDP provides extra information (ports numbering).

As used herein, the term “transport protocol” may refer to communicationprotocols that transport data between logical endpoints. A transportprotocol may include encapsulation protocol functionality.

Both TCP and UDP are commonly layered over an Internet Protocol (IP) 110for transmission. IP is a connectionless protocol for use onpacket-switched networks that provides a “best effort delivery”. Besteffort delivery does not guarantee delivery, nor does it assure propersequencing or avoidance of duplicate delivery. Generally these aspectsare addressed by TCP or another transport protocol based on UDP.

As a brief aside, consider a web browser that opens a webpage; the webbrowser application would generally open a number of network sockets todownload and/or interact with the various digital assets of the webpage(e.g., for a relatively common place webpage, this could entailinstantiating ˜300 sockets). The web browser can write (or read) data tothe socket; thereafter, the socket object executes system calls withinkernel space to copy (or fetch) data to data structures in the kernelspace.

As used herein, the term “domain” may refer to a self-contained memoryallocation e.g., user space, kernel space. A “domain crossing” may referto a transaction, event, or process that “crosses” from one domain toanother domain. For example, writing to a network socket from the userspace to the kernel space constitutes a domain crossing access.

In the context of a Berkeley Software Distribution (BSD) basednetworking implementation, data that is transacted within the kernelspace is stored in memory buffers that are also commonly referred to as“mbufs”. Each mbuf is a fixed size memory buffer that is usedgenerically for transfers (mbufs are used regardless of the callingprocess e.g., TCP, UDP, etc.). Arbitrarily sized data can be split intomultiple mbufs and retrieved one at a time or (depending on systemsupport) retrieved using “scatter-gather” direct memory access (DMA)(“scatter-gather” refers to the process of gathering data from, orscattering data into, a given set of buffers). Each mbuf transfer isparameterized by a single identified mbuf.

Notably, each socket transfer can create multiple mbuf transfers, whereeach mbuf transfer copies (or fetches) data from a single mbuf at atime. As a further complication, because the socket spans both: (i) userspace (limited privileges) and (ii) kernel space (privileged withoutlimitation), the socket transfer verifies that each mbuf copy into/outof kernel space is valid. More directly, the verification processensures that the data access is not malicious, corrupted, and/ormalformed (i.e., that the transfer is appropriately sized and is to/froman appropriate area).

The processing overhead associated with domain crossing is a non-trivialprocessing cost. Processing cost affects user experience both directlyand indirectly. A processor has a fixed amount of processing cyclesevery second; thus cycles that are used for transfer verificationdetract from more user perceptible tasks (e.g., rendering a video oraudio stream). Additionally, processor activity consumes power; thus,increases in processing overhead increases power consumption.

Referring back to FIG. 1, in addition to the generic TCP 104, UDP 106,and IPC 108 communication suites, the illustrated socket 102 also mayprovide access to various kernel extensions 112. A kernel extension is adynamically loaded bundle of executable code that executes from kernelspace. Kernel extensions may be used to perform low-level tasks thatcannot be performed in user space. These low-level tasks typically fallinto one or more of: low-level device drivers, network filters, and/orfile systems. Examples of sockets and/or extensions include withoutlimitation: route (IP route handling), ndry (packet 802.1X handling),key (key management), unix (translations for Unix systems), kernelcontrol, kernel events, parental controls, intrusion detection, contentfiltering, hypervisors, and/or any number of other kernel tasking.

Kernel extensions and public APIs enable, for example, 3^(rd) partysoftware developers to develop a wide variety of applications that caninteract with a computer system at even the lowest layers ofabstraction. For example, kernel extensions can enable socket levelfiltering, IP level filtering, and even device interface filtering. Inthe current consumer applications space, many emerging technologies nowrely on closely coupled interfaces to the hardware and kernelfunctionality. For example, many security applications “sniff” networktraffic to detect malicious traffic or filter undesirable content; thisrequires access to other application sandboxes (a level of privilegethat is normally reserved for the kernel).

Unfortunately, 3^(rd) party kernel extensions can be dangerous and/orundesirable. As previously noted, software applications are restrictedfor security and stability reasons; however the kernel is largelyunrestricted. A 3^(rd) party kernel extension can introduce instabilityissues because the 3rd party kernel extensions run in the same addressspace as the kernel itself (which is outside the purview of traditionalmemory read/write protections based on memory allocations). Illegalmemory accesses can result in segmentation faults and memorycorruptions. Furthermore, unsecure kernel extension can create securityvulnerabilities that can be exploited by malware. Additionally, evenwhere correctly used, a kernel extension can expose a user's data to the3^(rd) party software developer. This heightened level of access mayraise privacy concerns (e.g., the 3^(rd) party developer may have accessto browsing habits, etc.).

Existing Performance Optimization Technologies—

FIG. 2 illustrates one logical representation of a computer system thatimplements Input/Output (I/O) network control, useful for explainingvarious aspects of traditional network optimization. As depictedtherein, a software application 202 executing from user space opensmultiple sockets 204 to communicate with e.g., a web server. Each of thesockets interfaces with a Data Link Interface Layer (DLIL) 206.

The DLIL 206 provides a common interface layer to each of the variousphysical device drivers which will handle the subsequent data transfer(e.g., Ethernet, Wi-Fi, cellular, etc.). The DLIL performs a number ofsystem-wide holistic network traffic management functions. In one suchimplementation, the DLIL is responsible for BSD Virtual Interfaces,IOKit Interfaces (e.g., DLIL is the entity by which IOKit based networkdrivers are connected to the networking stack), Active Queue Management(AQM), flow control and advisory action, etc. In most cases, the devicedriver 208 may be handled by an external device (e.g., a basebandco-processor), thus the DLIL 206 is usually (but not always) the lowestlayer of the network communication stack.

During normal operation, the computer system will logically segment itstasks to optimize overall system operation. In particular, a processorwill execute a task, and then “context switch” to another task, therebyensuring that any single process thread does not monopolize processorresources from start to finish. More directly, a context switch is theprocess of storing the state of a process, or of a thread, so that itcan be restored and execution resumed from the same point later. Thisallows multiple processes to share a single processor. However,excessive amounts of context switching can slow processor performancedown. Notably, while the present discussion is primarily discussedwithin the context of a single processor for ease of understanding,multi-processor systems have analogous concepts (e.g., multipleprocessors also perform context switching, although contexts may notnecessarily be resumed by the same processor).

For example, consider the following example of a packet reception.Packets arrive at the device driver 208A. The hardware managed by thedevice driver 208A may notify the processor via e.g., a doorbell signal(e.g., an interrupt). The device driver 208A work loop thread handlesthe hardware interrupt/doorbell, then signals the DLIL thread (Loop 1210). The processor services the device driver 208A with high priority,thereby ensuring that the device driver 208A operation is notbottlenecked (e.g., that the data does not overflow the device driver'smemory and/or that the device driver does not stall). Once the data hasbeen moved out of the device driver, the processor can context switch toother tasks.

At a later point, the processor can pick up the DLIL 206 executionprocess again. The processor determines which socket the packets shouldbe routed to (e.g., socket 204A) and routes the packet dataappropriately (Loop 2 212). During this loop, the DLIL thread takes eachpacket, and moves each one sequentially into the socket memory space.Again, the processor can context switch to other tasks so as to ensurethat the DLIL task does not block other concurrently executedprocessing.

Subsequently thereafter, when the socket has the complete packet datatransfer the processor can wake the user space application and deliverthe packet into user space memory (Loop 3 214). Generally, user spaceapplications are treated at lower priority than kernel tasks; this canbe reflected by larger time intervals between suspension and resumption.While the foregoing discussion is presented in the context of packetreception, artisans of ordinary skill in the related arts will readilyappreciate, given the contents of the present disclosure, that theprocess is substantially reversed for packet transmission.

As demonstrated in the foregoing example, context switching ensures thattasks of different processing priority are allocated commensurateamounts of processing time. For example, a processor can spendsignificantly more time executing tasks of relatively high priority, andservice lower priority tasks on an as-needed basis. As a brief aside,human perception is much more forgiving than hardware operation.Consequently, kernel tasks are generally performed at a much higherpriority than user space applications. The difference in prioritybetween kernel and user space allows the kernel to handle immediatesystem management (e.g., hardware interrupts, and queue overflow) in atimely manner, with minimal noticeable impact to the user experience.

Moreover, FIG. 2 is substantially representative of every implementationof the traditional network communications stack. While implementationsmay vary from this illustrative example, virtually all networking stacksshare substantially the same delivery mechanism. The traditional networkcommunications stack schema (such as the BSD architecture andderivatives therefrom) have been very popular for the past 30 years dueto its relative stability of implementation and versatility across manydifferent device platforms. For example, the Assignee hereof hasdeveloped and implemented the same networking stack across virtually allof its products (e.g., MacBook®, iMac®, iPad®, and iPhone®, AppleWatch®, etc.).

Unfortunately, changing tastes in consumer expectations cannot beeffectively addressed with the one-size-fits-all model and theconservative in-kernel traditional networking stack. Artisans ofordinary skill in the related arts will readily appreciate, given thecontents of the present disclosure, that different device platforms havedifferent capabilities; for example, a desktop processor hassignificantly more processing and memory capability than a mobile phoneprocessor. More directly, the “one-size-fits-all” solution does notaccount for the underlying platform capabilities and/or applicationrequirements, and thus is not optimized for performance. Fine-tuning thetraditional networking stack for performance based on various “tailored”special cases results in an inordinate amount of software complexitywhich is untenable to support across the entire ecosystem of devices.

Emerging Use Cases—

FIG. 3 illustrates a logical block diagram of one exemplaryimplementation of Transport Layer Security (TLS) (the successor toSecure Sockets Layer (SSL)), useful to explain user/kernel spaceintegration complexities of emerging use cases.

As shown, an application executing from user space can open a HypertextTransfer Protocol (HTTP) session 302 with a TLS security layer 304 inorder to securely transfer data (Application Transport Security (ATS)services) over a network socket 306 that offers TCP/IP transport 308,310.

As a brief aside, TLS is a record based protocol; in other words, TLSuses data records which are arbitrarily sized (e.g., up to 16kilobytes). In contrast, TCP is a byte stream protocol (i.e., a byte hasa fixed length of eight (8) bits). Consequently, the TCP layersubdivides TLS records into a sequentially ordered set of bytes fordelivery. The receiver of the TCP byte stream reconstructs TLS recordsfrom the TCP byte stream by receiving each TCP packet, re-ordering thepackets according to sequential numbering to recreate the byte stream,and extracting the TLS record from the aggregated byte stream. Notably,every TCP packet of the sequence must be present before the TLS recordcan be reconstructed. Even though TCP can provide reliable deliveryunder lossy network conditions, there are a number of situations whereTLS record delivery could fail. For example, under ideal conditions TCPisolates packet loss from its client (TLS in this example), and a singleTCP packet loss should not result in failed TLS record delivery.However, the TLS layer or the application above may incorporate atimeout strategy in a manner that is unaware of the underlying TCPconditions. Thus, if there's significant packet loss in the network, theTLS timeout may be hit (and thus result in a failure to the application)even though TCP would normally provide reliable delivery.

Referring back to FIG. 3, virtually every modern operating systemexecutes TLS from user space when e.g., securely connecting to othernetwork entities, inter alia, a web browser instance and a server. Butexisting implementations of TLS are not executed from the kernel (orother privileged software layer) due to e.g., the complexity of errorhandling within the kernel. However, as a practical matter, TLS wouldoperate significantly better with information regarding the currentnetworking conditions (held in the kernel).

Ideally, the TLS layer should set TLS record sizes based on networkcondition information. In particular, large TLS records can efficientlyuse network bandwidth, but require many successful TCP packetdeliveries. In contrast, small TLS records incur significantly morenetwork overhead, but can survive poor bandwidth conditions.Unfortunately, networking condition information is lower layerinformation that is available to the kernel space (e.g., the DLIL anddevice drivers), but generally restricted from user space applications.Some 3^(rd) party application developers and device manufacturers haveincorporated kernel extensions (or similar operating systemcapabilities) to provide network condition information to the TLS userspace applications; however, kernel extensions are undesirable due tothe aforementioned security and privacy concerns. Alternately, some3^(rd) party applications infer the presence of lossy network conditionsbased on historic TLS record loss. Such inferences are an indirectmeasure and significantly less accurate and lag behind real-timeinformation (i.e., previous packet loss often does not predict futurepacket loss).

FIG. 4 illustrates a logical block diagram of an exemplaryimplementation of a Virtual Private Network (VPN), useful to explainrecursive/cross-layer protocol layer complexities of emerging use cases.

As shown, an application executing from user space can open a VirtualPrivate Network (VPN) session 402 over a network socket 406 that offersTCP/IP transport 408, 410. The VPN session is secured with EncapsulatingSecurity Protocol (ESP) 412. The encrypted packet is securely tunneledvia TLS 404 (in user space) and recursively sent again over TCP/IPtransport 408, 410.

As illustrated within FIG. 4, the exemplary VPN tunnel starts in userspace, crosses into kernel space, returns back to user space, and thencrosses back into kernel space before being transferred. Each of thedomain crossings results in costly context switches and data shufflingboth of which are processor intensive and inefficient. More directly,every time data traverses from user space to kernel space, the data mustbe validated (which takes non-trivial processing time). Additionally,context switching can introduce significant latency while the task issuspended.

Artisans of ordinary skill in the related arts, given the contents ofthe present disclosure, will readily appreciate that the exemplaryrecursive cross layer transaction of FIG. 4 is merely illustrative of abroad range of applications which use increasingly exotic protocol layercompositions. For example, applications that traverse the applicationproxy/agent data path commonly require tunneling TCP (kernel space) overapplication proxy/agent data path (user space) over UDP/IP (kernelspace). Another common implementation is IP (kernel space) over QuickUDP Internet Connections (QUIC) (user space) over UDP/IP (kernel space).

FIG. 5 illustrates a logical block diagram of an exemplaryimplementation of application based tuning, useful to explain variousother workload optimization complexities of emerging use cases.

As shown, three (3) different concurrently executed applications (e.g.,a real time application 502, interactive application 504, and filetransfer applications 506) in user space, each open a session overnetwork sockets 508 (508A, 508B, 508C) that offer TCP/UDP/IP transport510/512. Depending on the type of physical interface required, thesessions are switched to BSD network interfaces (ifnet) 514 (514A, 514B,514C) which handle the appropriate technology. Three differentillustrated technology drivers are shown: Wi-Fi 516, Bluetooth 518, andcellular 520.

It is well understood within the networking arts that differentapplication types are associated with different capabilities andrequirements. One such example is real time applications 502, commonlyused for e.g., streaming audio/visual and/or other “live” data. Realtime data has significant latency and/or throughput restrictions;moreover, certain real time applications may not require (and/orsupport) retransmission for reliable delivery of lost or corrupted data.Instead, real time applications may lower bandwidth requirements tocompensate for poor transmission quality (resulting in lower quality,but timely, delivered data).

Another such example is interactive applications 504, commonly used fore.g., human input/output. Interactive data should be delivered atlatencies that are below the human perceptible threshold (within severalmilliseconds) to ensure that the human experience is relativelyseamless. This latency interval may be long enough for a retransmission,depending on the underlying physical technology. Additionally, humanperception can be more or less tolerant of certain types of datacorruptions; for example, audio delays below 20 ms are generallyimperceptible, whereas audio corruptions (pops and clicks) arenoticeable. Consequently, some interactive applications may allow forsome level of error correction and/or adopt less aggressive bandwidthmanagement mechanisms depending on the acceptable performancerequirements for human perception.

In contrast to real time applications and interactive applications, filetransfer applications 506 require perfect data fidelity without latencyrestrictions. To these ends, most file transfer technologies supportretransmission of lost or corrupted data, and retransmission can haverelatively long attempt intervals (e.g., on the order of multipleseconds to a minute).

Similarly, within the communication arts, different communicationtechnologies are associated with different capabilities andrequirements. For example, Wi-Fi 516 (wireless local area networkingbased on IEEE 802.11) is heavily based on contention based access and isbest suited for high bandwidth deliveries with reasonable latency. Wi-Fiis commonly used for file transfer type applications. Bluetooth 518(personal area networking) is commonly used for low data rate and lowlatency applications. Bluetooth is commonly used for human interfacedevices (e.g., headphones, keyboards, and mouses). Cellular networktechnologies 520 often provide non-contention based access (e.g.,dedicated user access) and can be used over varying geographic ranges.Cellular voice or video delivery is a good example of streaming dataapplications. Artisans of ordinary skill in the related arts willreadily recognize that the foregoing examples are purely illustrative,and that different communication technologies are often used to supporta variety of different types of application data. For example, Wi-Fi 516can support file transfer, real time data transmission and/orinteractive data with equivalent success.

Referring back to FIG. 5, the presence of multiple concurrentlyexecuting applications of FIG. 5 (real time application 502, interactiveapplication 504, and file transfer applications 506) illustrates thecomplexities of multi-threaded operation. As shown therein, theexemplary multi-threaded operation incurs a number of server loops. Eachserver loop represents a logical break in the process during which theprocessor can context switch (see also aforementioned discussion ofExisting Performance Optimization Technologies, and corresponding FIG.2).

Moreover, in the computing arts, a “locking” synchronization mechanismis used by the kernel to enforce access limits (e.g., mutual exclusion)on resources in multi-threaded execution. During operation, each threadacquires a lock before accessing the corresponding locked resourcesdata. In other words, at any point in time, the processor is necessarilylimited to only the resources available to its currently executingprocess thread.

Unfortunately, each of the applications has different latency,throughput and processing utilization requirements. Since, each of thenetwork interfaces is sending and receiving data at different times, indifferent amounts, and with different levels of priority. From a purelylogistical standpoint, the kernel is constantly juggling between highpriority kernel threads (to ensure that the high priority hardwareactivities do not stall out) while still servicing each of itsconcurrently running applications to attempt to provide acceptablelevels of service. In some cases, however, the kernel is bottlenecked bythe processor's capabilities. Under such situations, some threads willbe deprioritized; currently, the traditional networking stackarchitecture is unable it clearly identify which threads can bedeprioritized while still providing acceptable user service.

For example, consider an “expected use” device of FIG. 5; the processoris designed for the expected use case of providing streaming video.Designing for expected use cases allows the device manufacturer to useless capable, but adequate components thereby reducing bill of materials(BOM) costs and/or offering features at a reasonable price point forconsumers. In this case, a processor is selected that nominally meetsthe requirements for a streaming video application that is receivingstreaming video data via one of the network interfaces (e.g., the Wi-Fiinterface), and constantly servicing the kernel threads associated withit. Rendering the video with a real time application 502 from thereceived data is a user space application that is executed concurrentlybut at a significantly lower priority. During expected usage, the videorendering is adequate.

Unfortunately, the addition of an unexpected amount of additionalsecondary interactive applications 504 (e.g., remote control interface,headphones, and/or other interface devices) and/or background filetransfer applications can easily overwhelm the processor. Specifically,the primary real time application does not get enough CPU cycles to runwithin its time budget, because the kernel threads handling networkingare selected at a higher priority. In other words, the user spaceapplication is not able to depress the priority of kernel networkingthreads (which are servicing both the primary and secondary processes).This can result in significantly worse user experience when the videorendering stalls out (video frame misses or video frame drops); whereassimply slowing down a file transfer or degrading the interactioninterface may have been preferable.

Prior art solutions have tailored software for specific deviceimplementations (e.g., the Apple TV®). For example, the device can bespecifically programmed for an expected use. However, tailored solutionsare becoming increasingly common and by extension the exceptions haveswallowed the more generic use case. Moreover, tailored solutions areundesirable from multiple software maintenance standpoints. Devices havelimited productive lifetimes, and software upkeep is non-trivial.

Ideally, a per-application or per-profile workload optimization wouldenable a single processor (or multiple processors) to intelligentlydetermine when and/or how too intelligently context switch and/orprioritize its application load (e.g., in the example of FIG. 5, toprioritize video decode). Unfortunately, such solutions are not feasiblewithin the context of the existing generic network sockets and genericnetwork interfaces to a monolithic communications stack.

Exemplary Networking Architecture—

A networking stack architecture and technology that caters to the needsof non-kernel based networking use cases is disclosed herein. Unlikeprior art monolithic networking stacks, the exemplary networking stackarchitecture described hereinafter includes various components that spanmultiple domains (both in-kernel, and non-kernel), with varyingtransport compositions, workload characteristics and parameters.

In one exemplary embodiment, a networking stack architecture isdisclosed that provides an efficient infrastructure to transfer dataacross domains (user space, non-kernel, and kernel). Unlike thetraditional networking paradigm that hide the underlying networkingtasks within the kernel and substantially limits control thereof by anynon-kernel applications, the various embodiments described herein enablefaster and more efficient cross domain data transfers.

Various embodiments of the present disclosure provide a faster and moreefficient packet input/output (I/O) infrastructure than prior arttechniques. Specifically, unlike traditional networking stacks that usea “socket” based communication, disclosed embodiments can transfer datadirectly between the kernel and user space domains. Direct transferreduces the per-byte and per-packet costs relative to socket basedcommunication. Additionally, direct transfer can improve observabilityand accountability with traffic monitoring.

In one such variant, a simplified data movement model that does notrequire mbufs (memory buffers) is described in greater detail herein.During one such exemplary operation, the non-kernel processes canefficiently transfer packets directly to and from the in-kernel drivers.

In another embodiment, a networking stack architecture is disclosed thatexposes the networking protocol stack infrastructure to user spaceapplications via network extensions. In one such embodiment, the networkextensions are software agents that enable extensible,cross-platform-capable, user space control of the networking protocolstack functionality. In another such embodiment, an in-process userspace networking stack facilitates tighter integration between theprotocol layers (including TLS) and the application or daemon. In somecases, the user space architecture can expose low-level networkinginterfaces to transport protocols and/or encapsulation protocols such asUDP, TCP, and QUIC; and enable network protocol extensions and rapiddevelopment cycles. Moreover, artisans of ordinary skill in the relatedarts, given the contents of the present disclosure, will readilyappreciate that the various principles described herein may be appliedto a variety of other operating systems (such as Windows, Linux, Unix,Android), and/or other cross platform implementations.

In some variants, exemplary embodiments of the networking stack cansupport multiple system-wide networking protocol stack instances(including an in-kernel traditional network stack). Specifically, in onesuch variant, the exemplary networking stack architecture coexists withthe traditional in-kernel networking stack so as to preserve backwardscompatibility for legacy networking applications. In suchimplementations, the in-kernel network stack instance can coexist withthe non-kernel network stack via namespace sharing and flow forwarding.

As used herein, an “instance” may refer to a single copy of a softwareprogram or other software object; “instancing” and “instantiations”refers to the creation of the instance. Multiple instances of a programcan be created; e.g., copied into memory several times. Software objectinstances are instantiations of a class; for example, a first softwareagent and second software instance are each distinct instances of thesoftware agent class.

In one such implementation, load balancing for multiple networkingstacks is handled within the kernel, thereby ensuring that no singlenetworking stack (including the in-kernel stack) monopolizes systemresources.

As a related variant, current/legacy applications can be handled withinthe in-kernel stack. More directly, by supporting a separate independentin-kernel B S D stack, legacy applications can continue to work withoutregressions in functionality and performance.

FIG. 6 illustrates one logical representation of an exemplary networkingstack architecture, in accordance with the various aspects of thepresent disclosure. While the system depicts a plurality of user spaceapplications 602 and/or legacy applications 612, artisans of ordinaryskill will readily appreciate given the contents of present disclosurethat the disclosed embodiments may be used within single applicationsystems with equivalent success.

As shown, a user space application 602 can initiate a network connectionby instancing user space protocol stacks 604. Each user space protocolstacks includes network extensions for e.g., TCP/UDP/QUIC/IP,cryptography, framing, multiplexing, tunneling, and/or any number ofother networking stack functionalities. Each user space protocol stack604 communicates with one or more nexuses 608 via a channel input/output(I/O) 606. Each nexus 608 manages access to the network drivers 610.Additionally shown is legacy application 612 support via existingnetwork socket technologies 614. While the illustrated embodiment showsnexus connections to both user space and in-kernel networking stacks, itis appreciated that the nexus may also enable e.g., non-kernelnetworking stacks (such as may be used by a daemon or other non-kernel,non-user process).

The following topical sections hereinafter describe the salient featuresof the various logical constructs in greater detail.

Exemplary I/O Infrastructure—

In one exemplary embodiment, the non-kernel networking stack provides adirect channel input output (I/O) 606. In one such implementation, thechannel I/O 606 is included as part of the user space protocol stack604. More directly, the channel I/O 606 enables the delivery of packetsas a raw data I/O into kernel space with a single validation (e.g., onlywhen the user stack provides the data to the one or more nexuses 608).The data can be directly accessed and/or manipulated in situ, the dataneed not be copied to an intermediary buffer.

In one exemplary implementation, a channel is an I/O scheme leveragingkernel-managed shared memory. During an access, the channel I/O ispresented to the process (e.g., the user process or kernel process) as afile descriptor based object, rather than as data. In order to accessthe data, the process de-references the file descriptor for directaccess to the shared memory within kernel space. In one suchimplementation, the file descriptor based object based I/O is compatiblewith existing operating system signaling and “eventing” (eventnotification/response) mechanisms. In one exemplary variant, the channelI/O is based on Inter Process Communication (IPC) packets.

As used herein, the term “descriptor” may refer to data structures thatindicate how other data is stored. Descriptors generally includemultiple parameters and can be used to identify more complex datastructures; for example, a descriptor may include one or more of type,size, address, tag, flag, headers, footers, metadata, structural linksto other data descriptors or locations, and/or any other number offormat or construction information.

Within the context of the present disclosure, as used herein, the term“pointer” may refer to a specific reference data type that “points” or“references” a location of data in memory. Typically, a pointer stores amemory address that is interpreted by a compiler as an absolute locationin system memory or a relative location in system memory based on e.g.,a base address, reference address, memory window, or other memorysubset. During operation, a pointer is “de-referenced” to recover thedata that is stored in the location of memory.

As used herein, the term “metadata” refers to data that describes data.Metadata varies widely in application, but generally falls into one ofthe descriptive, structural, and/or administrative categories.Descriptive metadata describes data in a manner to enable e.g.,discovery and/or identification. Common examples include withoutlimitation e.g., type, size, index tags, and keywords. Structuralmetadata describes the structure of the data e.g., how compound objectsare put together. Common examples include without limitation e.g.,prefix, postfix, table of contents, order, and/or any other informationthat describes the relationships and other characteristics of digitalmaterials. Administrative metadata provides information to help manage aresource; common examples include e.g., authorship and creationinformation, access privileges, and/or error checking and security basedinformation (e.g., cyclic redundancy checks (CRC), parity, etc.).

In one exemplary embodiment, the channel I/O can be further leveraged toprovide direct monitoring of its corresponding associated memory. Moredirectly, unlike existing data transfers which are based on mbuf baseddivide/copy/move, etc., the channel I/O can provide (with appropriateviewing privileges) a direct window into the memory accesses of thesystem. Such implementations further simplify software development asdebugging and/or traffic monitoring can be performed directly ontraffic. Direct traffic monitoring can reduce errors attributed to falsepositives/false negatives caused by e.g., different software versioning,task scheduling, compiler settings, and/or other software introducedinaccuracies.

More generally, unlike prior art solutions which relied on specializednetworking stack compositions to provide different degrees of visibilityat different layers, the monitoring schemes of the present disclosureprovide consistent system-wide channel monitoring infrastructures.Consistent frameworks for visibility, accounting, and debugging greatlyimprove software maintenance and upkeep costs.

Additionally, simplified schemes for egress filtering can be used toprevent traffic spoofing for user space networking stack instances. Forexample, various embodiments ensure that traffic of an applicationcannot be hijacked by another malicious application (by the latterclaiming to use the same tuple information, e.g. TCP/UDP port).

In one exemplary embodiment, the in-kernel network device drivers (e.g.Wi-Fi, Cellular, Ethernet) use simplified data movement models based onthe aforementioned channel I/O scheme. More directly, the user spacenetworking stacks can directly interface to each of the variousdifferent technology based network drivers via channel I/O; in thismanner, the user space networking stacks do not incur the traditionaldata mbuf based divide/copy/move penalties. Additionally, user spaceapplications can directly access user space networking components forimmediate traffic handling and processing.

Exemplary Nexus

In one exemplary embodiment, the networking stack connects to one ormore nexus 608. In one such implementation, the nexus 608 is a kernelspace process that arbitrates access to system resources including,without limitation e.g., shared memory within kernel space, networkdrivers, and/or other kernel or user processes. In one such variant, thenexus 608 aggregates one or more channels 606 together for access to thenetwork drivers 610 and/or shared kernel space memory.

In one exemplary implementation, a nexus is a kernel process thatdetermines the format and/or parameters of the data flowing through itsconnected channels. In some variants, the nexus may further performingress and/or egress filtering.

The nexus may use the determined format and/or parameter information tofacilitate one-to-one and one-to-many topologies. For example, the nexuscan create user-pipes for process-to-process channels; kernel-pipes forprocess-to-kernel channels; network interfaces for direct channelconnection from a process to in-kernel network drivers, or legacynetworking stack interfaces; and/or flow-switches for multiplexing flowsacross channels (e.g., switching a flow from one channel to one or moreother channels).

Additionally, in some variants the nexus may provide the format,parameter, and/or ingress egress information to kernel processes and/orone or more appropriately privileged user space processes.

In one exemplary embodiment, the nexus 608 may additionally ensure thatthere is fairness and/or appropriately prioritize each of its connectedstacks. For example, within the context of FIG. 6, the nexus 608balances the network priorities of both the existing user spaceapplication networking stacks 604, as well as providing fair access forlegacy socket based access 614. For example, as previously alluded to,existing networking stacks could starve user space applications becausethe kernel threads handling the legacy networking stack operated athigher priorities than user space applications. However, the exemplarynexus 608 ensures that legacy applications do not monopolize systemresources by appropriately servicing the user space network stacks aswell as the legacy network stack.

In one such embodiment, in-kernel, non-kernel, and/or user spaceinfrastructures ensure fairness and can reduce latency due to e.g.,buffer bloat (across channels in a given nexus, as well as flows withina channel). In other words, the in-kernel and/or user spaceinfrastructures can negotiate proper buffering sizes based on theexpected amount of traffic and/or network capabilities for each flow. Bybuffering data according to traffic and/or network capability, buffersare not undersized or oversized.

As a brief aside, “buffer bloat” is commonly used to describe e.g., highlatency caused by excessive buffering of packets. Specifically, bufferbloat may occur when excessively large buffers are used to support areal time streaming application. As a brief aside, TCP retransmissionmechanism relies on measuring the occurrence of packet drops todetermine the available bandwidth. Under certain congestion conditions,excessively large buffers can prevent the TCP feedback mechanism fromcorrectly inferring the presence of a network congestion event in atimely manner (the buffered packets “hide” the congestion, since theyare not dropped). Consequently, the buffers have to drain before TCPcongestion control resets and the TCP connection can correct itself.

Referring back to FIG. 6, in one exemplary embodiment, Active QueueManagement (AQM) can be implemented in the kernel across one or more(potentially all) of the flow-switch clients (user space and in-kernelnetworking stack instances). AQM refers to the intelligent culling ofnetwork packets associated with a network interface, to reduce networkcongestion. By dropping packets before the queue is full, the AQMensures no single buffer approaches its maximum size, and TCP feedbackmechanisms remain timely (thereby avoiding the aforementioned bufferbloat issues).

While the foregoing example is based on “fairness” standard, artisans ofordinary skill in the related arts will readily appreciate that otherschemes may be substituted with equivalent success given the contents ofthe present disclosure. For example, some embodiments may dynamically orstatically service the user application networking space with greater orless weight compared to the legacy socket based access. For example,user application networking space may be more heavily weighted toimprove overall performance or functionality, whereas legacy socketbased access may be preferred where legacy applications arepreferentially supported (e.g., see Protocol Unloading Offloading,discussed infra).

Exemplary Network Extensions

In one exemplary embodiment of the present disclosure, a networkextension is disclosed. A network extension is an agent-based extensionthat is tightly coupled to network control policies. The agent isexecuted by the kernel and exposes libraries of network controlfunctionality to user space applications. During operation, user spacesoftware can access kernel space functionality through the context andprivileges of the agent.

As used herein, the term “agent” may refer to a software agent that actsfor a user space application or other program in a relationship ofagency with appropriate privileges. The agency relationship between theagent and the user space application implies the authority to decidewhich, if any, action is appropriate given the user application andkernel privileges. A software agent is privileged to negotiate with thekernel and other software agents regarding without limitation e.g.,scheduling, priority, collaboration, visibility, and/other sharing ofuser space and kernel space information. While the agent negotiates withthe kernel on behalf of the application, the kernel ultimately decideson scheduling, priority, etc.

Various benefits and efficiencies can be gained through the use ofnetwork extensions. In particular, user space applications can controlthe protocol stack down to the resolution of exposed threads (i.e., thethreads that are made available by the agent). In other words, softwareagents expose specific access to lower layer network functionality whichwas previously hidden or abstracted away from user space applications.For example, consider the previous examples of TLS record sizing (seee.g., FIG. 3, and related discussion); by exposing TCP networkconditions to the TLS application within the user space, the TLSapplication can correctly size records for network congestion and/orwait for underlying TCP retransmissions (rather than timing out).

Similarly, consider the previous examples of multi-threading within thecontext of expected use devices (see e.g., FIG. 5, and relateddiscussion); the primary user space application (e.g., video coding) andadditional secondary interactive applications (e.g., remote controlinterface, headphones, and/or other interface devices) can internallynegotiate their relative priority to the user's experience. The userspace applications can appropriately adjust their priorities for thenexus (i.e., which networking threads are serviced first and/or shouldbe deprioritized). Consequently, the user space applications candeprioritize non-essential network accesses, thereby preserving enoughCPU cycles for video decode.

As a related benefit, since a software agent represents the applicationto the kernel; the agent can trust the kernel, but the kernel may or maynot trust the agent. For example, a software agent can be used by thekernel to convey network congestion information in a trusted manner tothe application; similarly, a software agent can be used by anapplication to request a higher network priority. Notably, since asoftware agent operates from user space, the agent's privilege is notpromoted to kernel level permissions. In other words, the agent does notpermit the user application to exceed its privileges (e.g., the agentcannot commandeer the network driver at the highest network priority, orforce a read/write to another application's memory space without theother kernel and/or other application's consent).

Networking extensions allow the user space application to executenetworking communications functionality within the user space andinterpose a network extension between the user space application and thekernel space. As a result, the number of cross domain accesses forcomplex layering of different protocol stacks can be greatly reduced.Limiting cross domain accesses prevents context switching and allows theuser space to efficiently police its own priorities. For example,consider the previous example of a VPN session as was previouslyillustrated in FIG. 4. By keeping the TCP/IP, Internet Protocol Security(IPsec) and TLS operations within user space, the entire tunnel can beperformed within the user space, and only cross the user/kernel domainonce.

As used herein, the term “interposition” may refer to the insertion ofan entity between two or more layers. For example, an agent isinterposed between the application and the user space networking stack.Depending on the type of agent or network extension, the interpositioncan be explicit or implicit. Explicit interposition occurs where theapplication explicitly instances the agent or network extension. Forexample, the application may explicitly call a user space tunnelextension. In contrast, implicit interposition occurs where theapplication did not explicitly instance the agent or network extension.Common examples of implicit interposition occur where one user spaceapplication sniffs the traffic or filters the content of another userspace application.

Namespace Sharing & Flow Forwarding Optimizations

In one exemplary optimization of the present disclosure, the nexusincludes a namespace registration and management component that managesa common namespace for all of its connected networking stack instances.As a brief aside, a namespace generally refers to a set of uniqueidentifiers (e.g., the names of types, functions, variables) within acommon context. Namespaces are used to prevent naming “collisions” whichoccur where multiple processes call the same resource differently and/orcall different resources the same.

In one such implementation, the shared networking protocol has a commonnamespace (e.g., {Address, Protocol, and Port}) across multiplenetworking stack instances. Sharing a namespace between differentnetworking stacks reduces the amount of kernel burden, as the kernel cannatively translate (rather than additionally adding a layer of networkaddress translation).

For example, if a first application acquires port 80, the namespaceregistration ensures that other applications will not use port 80 (e.g.,they can be assigned e.g., port 81, 82, etc.) In some suchimplementations, legacy clients may use default namespaces that conflict(e.g., a default web client may always select port 80); thus the sharednamespace registration may also be required to force a re-assignment ofa new identifier (or else translate for) such legacy applications.

In one exemplary embodiment, the namespace registration and managementcomponents control flow-switching and forwarding logic of eachflow-switch nexus instance. For example, as previously noted, the nexuscan create user-pipes for process-to-process channels; kernel-pipes forprocess-to-kernel channels; network interfaces for direct channelconnection from a process to in-kernel network drivers, or legacynetworking stack interfaces; and/or flow-switches for multiplexing flowsacross channels (e.g., switching a flow from one channel to one or moreother channels).

For example, during normal operation when an application requests aport, the namespace registration and management will create a flow andassign a particular port to the application. Subsequent packetsaddressed to the port will be routed appropriately to the flow'scorresponding application. In one such variant, packets that do notmatch any registered port within the shared namespace registration andmanagement will default to the legacy networking stack (e.g., theflow-switch assumes that the unrecognized packet can be parsed and/orignored by the fallback legacy stack).

Artisans of ordinary skill in the related arts will readily appreciate,given the contents of the present disclosure that disparate and/orotherwise distinct namespace registrations and/or management componentsmay be preferable based on other implementation specific considerations.For example, some implementations may prefer to shield namespaces fromother external processes e.g., for security and/or privacyconsiderations. In other implementations, the benefits associated withnative namespace translation may be less important than supportinglegacy namespaces.

Protocol Onloading and Offloading

In the foregoing discussions, the improvements to user space operationmay be primarily due to the user space networking stack, as shown inFIG. 6. However, various embodiments of the present disclosure alsoleverage the existing legacy host networking infrastructure to handlenetworking transactions which are unrelated to user experience.

Colloquially, the term “hardware offload” may be commonly used to denotetasks which can be handled within dedicated hardware logic to improveoverall processing speed or efficiency. One such example is the cyclicredundancy check (CRC) calculation which is an easily parameterized,closed, iterative calculation. The characteristics of CRC calculationlend itself to hardware offload because the CRC does not benefit fromthe flexibility of a general purpose processor, and CRC calculations arespecialized functions that are not transferable to other processingoperations.

By analogous extension, as used herein, the term “protocol offload” mayrefer to processes that should be handled within the legacy networkingstack because they are not specific to a user space application or task.In contrast, the term “protocol onload” may refer to processes thatshould be handled within a user space networking stack because they arespecific to a user space application or task and benefit the overallperformance. As a general qualitative criteria, tasks which are “fast”(e.g., generally UDP/TCP/IP based user space applications) are protocolonloaded to improve user performance; in contrast “slow” tasks (e.g.,ARP, IPv6 Neighbor Discovery, Routing table updates, control path formanaging interfaces, etc.) are protocol offloaded.

For example, consider Address Resolution Protocol (ARP) requesthandling; when an ARP request comes in, the host processor responds witha reply. However, the ARP request is non-specific to a user spaceapplication; rather the ARP reply concerns the holistic system. Moregenerally, any networking process that is not specific to an applicationspace can be implemented within the kernel under legacy techniques.Alternatively, any process that can be handled regardless of devicestate should remain with the kernel (e.g., the kernel persists acrosslow power states, and is never killed).

By allowing the mature in-kernel networking stack to retain ownership ofcertain control logic (e.g. routing and policy table, interfaceconfiguration, address management), various embodiments of the presentdisclosure avoid “split-brain” behaviors. In other words, the kernelensures that networking data and/or availability remains consistentregardless of the user space application availability.

Exemplary User Space Networking Stack

Referring now to FIG. 7, one logical block diagram of an exemplary userspace networking stack 700 is depicted. As shown, the user spacenetworking stack 700 includes an application interface 702, and anoperating system interface 704. Additionally, the user space networkingstack includes one or more user space instances of TLS 706, QUIC 708,TCP 710, UDP 712, IP 714, and ESP 716. The disclosed instances arepurely illustrative, artisans of ordinary skill in the related arts willreadily appreciate that any other user space kernel extension and/orsocket functionality may be made available within the user spacenetworking stack 700.

In one exemplary embodiment, the user space networking stack 700 isinstantiated within an application user space 718. More directly, theuser space networking stack 700 is treated identically to any one ofmultiple threads 710 within the application user space 718. Each of thecoexisting threads 720 has access to the various functions and librariesoffered by the user space networking stack via a direct function call.

As a brief aside, each of the threads 720 reside within the same addressspace. By virtue of their shared addressability, each of the threads maygrant or deny access to their portions of shared address space viaexisting user space memory management schemes and/or virtual machinetype protections. Additionally, threads can freely transfer datastructures from one to the other, without e.g., incurring cross domainpenalties. For example, TCP data 710 can be freely passed to TLS 706 asa data structure within a user space function call.

As previously noted, the user space networking stack 700 may grant ordeny access to other coexistent user space threads; e.g., a user spacethread is restricted to the specific function calls and privileges madeavailable via the application interface 702. Furthermore, the user spacenetworking stack 700 is further restricted to interfacing the operatingsystem via the specific kernel function calls and privileges madeavailable via the operating system interface 704. In this manner, boththe threads and the user space networking stack have access andvisibility into the kernel space, without compromising the kernel'ssecurity and stability.

One significant benefit of the user space networking stack 700 is thatnetworking function calls can be made without acquiring various locksthat are present in the in-kernel networking stack. As previously noted,the “locking” mechanism is used by the kernel to enforce access limitson multiple threads from multiple different user space applications;however in the user space, access to shared resources are handled withinthe context of only one user application space at a time, consequentlyaccess to shared resources are inherently handled by the singlethreading nature of user space execution. More directly, only one threadcan access the user space networking stack 700 at a time; consequently,kernel locking is entirely obviated by the user space networking stack.

Another benefit of user space based network stack operation is crossplatform compatibility. For example, certain types of applications(e.g., iTunes®, Apple Music® developed by the Assignee hereof) aredeployed over a variety of different operating systems. Similarly, someemerging transport protocols (e.g. QUIC) are ideally served by portableand common software between the client and server endpoints. Consistencyin the user space software implementation allows for better and moreconsistent user experience, improves statistical data gathering andanalysis, and provides a foundation for enhancing, experimenting anddeveloping network technologies used across such services. In otherwords, a consistent user space networking stack can be deployed over anyoperating system platform without regard for the native operating systemstack (e.g., which may vary widely).

Another important advantage of the exemplary user space networking stackis the flexibility to extend and improve the core protocolfunctionalities, and thus deliver specialized stacks based on theapplication's requirements. For example, a video conferencingapplication (e.g., FaceTime® developed by the Assignee hereof) maybenefit from a networking stack catered to optimize performance forreal-time voice and video-streaming traffics (e.g., by allocating moreCPU cycles for video rendering, or conversely deprioritizing unimportantancillary tasks). In one such variant, a specialized stack can bedeployed entirely within the user space application, without specializedkernel extensions or changes to the kernel. In this manner, thespecialized user space networking stack can be isolated from networkingstacks. This is important both from a reliability standpoint (e.g.,updated software doesn't affect other software), as well as to minimizedebugging and reduce development and test cycle times.

Furthermore, having the network transport layer (e.g. TCP, QUIC) residein user space can open up many possibilities for improving performance.For example, as previously alluded to, applications (such as TLS) can bemodified depending on the underlying network connections. User spaceapplications can be collapsed or tightly integrated into networktransports. In some variants, data structure sizes can be adjusted basedon immediate lower layer network condition information (e.g., toaccommodate or compensate for poor network conditions). Similarly,overly conservative or under conservative transport mechanisms can beavoided (e.g., too much or not enough buffering previously present atthe socket layer). Furthermore, unnecessary data copies and/ortransforms can be eliminated and protocol signaling (congestion, error,etc.) can be delivered more efficiently.

In yet another embodiment, the exemplary user space networking stackfurther provides a framework for both networking clients and networkingproviders. In one such variant, the networking client framework allowsthe client to interoperate with any network provider (including thelegacy BSD stack). In one such variant, the network provider frameworkprovides consistent methods of discovery, connection, and data transferto networking clients. By providing consistent frameworks for clientsand providers which operate seamlessly over a range of differenttechnologies (such as a VPN, Bluetooth, Wi-Fi, cellular, etc.), theclient software can be greatly simplified while retaining compatibilitywith many different technologies.

Exemplary Proxy Agent Application Operation

FIG. 8 depicts one logical flow diagram useful to summarize theconvoluted data path taken for a prior art application using a proxyagent application within the context of the traditional networkingstack. As shown therein, an application 802 transmits data via a socket804A to route data packets to a proxy agent application 814 via a TCP/IP806/808 and a BSD network interface 810A. The data packets enter kernelspace; this is a first domain crossing which incurs validation andcontext switching penalties.

Inside the kernel, the data is divided/copied/moved for delivery via theTCP/IP stack 806/808 to the BSD network interface 810A. The BSD networkinterface 810A routes the data to a virtual driver 812A. These steps mayintroduce buffering delays as well as improper buffer sizing issues suchas buffer bloat.

In order to access the application proxy (which is in a different userspace), the virtual driver reroutes the data to a second socket 804Bwhich is in the different user space from the original application. Thisconstitutes a second domain crossing, which incurs additional validationand context switching penalties.

In user space, the data enters an agent 814 which prepares the data fordelivery (tunneling 816, framing 818, and cryptographic security 820).Thereafter, the proxy agent 814 transmits the prepared data via a socket804B to route data packets to a user space driver 822 via the TCP/IP806/808 and a separate BSD network interface 810B. Again, the data ispassed through the socket 804B. This is a third domain crossing, withvalidation and context switching penalties.

Inside the kernel, the data is divided/copied/moved for delivery via theTCP/IP stack 806/808 to a B S D network interface 810B. The steps of TheB S D network interface 810B routes the data to a virtual driver 812B.These steps introduce additional buffering delays as well as improperbuffer sizing issues such as buffer bloat.

Finally, the virtual driver 812B reroutes the data to the user spacedriver (e.g., a Universal Serial Bus (USB) driver), which requiresanother socket transfer from 804B to 804C; the data crosses into theuser space for the user based driver 822, and crosses the domain a fifthtime to be routed out the USB Hardware (H/W) driver 824. Each of thesedomain crossings are subject to the validation and context switchingpenalties as well as any buffering issues.

FIG. 9 depicts one logical flow diagram useful to summarize an exemplaryproxy agent application within the context of the user space networkingstack, in accordance with the various aspects of the present disclosure.

As shown therein, an application 902 provides data via shared memoryspace file descriptor objects to the agent 904. The agent 904 internallyprocesses the data via TCP/IP 906/908 to the tunneling function 910.Thereafter, the data is framed 912, cryptographically secured 914, androuted via TCP/IP 906/908 to the user driver 916. The user driver uses achannel I/O to communicate with nexus 918 for the one (and only) domaincrossing into kernel space. Thereafter, the nexus 918 provides the datato the H/W driver 920.

When compared side-by-side, the user space networking stack 900 has onlyone (1) domain crossing, compared to the traditional networking stack800 which crossed domains five (5) times for the identical VPNoperation. Moreover, each of the user space applications could directlypass data via function calls within user memory space between each ofthe intermediary applications, rather than relying on the kernel basedgeneric mbuf divide/copy/move scheme (and its associated bufferinginefficiencies).

Daemon Specific Considerations

As used herein, the term “daemon” refers to a special process that runswithin user space. Daemons run in the background and do not require anyuser interaction at all. Moreover, 3^(rd) party developers also do nothave control and cannot create system daemons. Only the Assignee hereofcan create daemons for its own systems; they are a special, privilegedtype of processes that 3^(rd) party developers cannot deploy. Daemonsare never suspended, and are usually limited to a fixed memoryallocation.

Under some circumstances, a networking daemon can accidentally leakmemory or cause other problems. For reasons previously articulatedabove, identifying rogue threads in traditional networking stacks wasunfeasible because existing monolithic networking stacks handled allnetwork connections. However, within the context of the presentdisclosure, networking daemons are treated as a thread associated withthe user space networking stack that called it (even though the daemonis not really a user process). Grouping daemons with their correspondinguser space networking stack can greatly mitigate the impact of daemonerrors and improve daemon recovery. In particular, the calling userspace networking stack is responsible for tracking its memory usage inthe daemons (the daemons are not shared with other processes); thusrogue daemons can be easily identified.

Unfortunately, user space daemons can be mistaken for runaway processesby the kernel because the kernel is not aware of user space daemonoperation. Consequently, solutions for managing a user space daemon'sbackground activities are needed.

Normally, in order to ensure that a daemon is correctly operating, thekernel sets a “high water mark” for a daemon's thread (an amount of datathat a daemon cannot exceed during normal usage.) Subsequently, if thedaemon's thread leaks memory, the process can be terminated and/orrestarted.

Unfortunately, a simple “high water mark” can pose problems fornetworking daemons. In particular, networking daemon processes may beinfrequently used, but consume a lot of memory during usage. Forexample, TCP packet handling usually queues packet buffers until theapplication is ready to consume the packets. Consequently, if theapplication does not read data for long periods of time (which isrelatively common for a network daemon), or if the network sends a largebatch of packets (e.g., TCP stores out-of-order segments in itsreassembly queue until missing segments arrive), then the TCP flow couldbe associated with a large amount of memory. The TCP protocol in thiscase runs within the daemon, and the memory associated with the TCP flowincreases the physical memory footprint of the daemon. This increasedphysical memory footprint could exceed a daemon's allowable high watermark. Consequently, the daemon could be wrongly targeted by the systemfor termination.

To these ends, various embodiments of the networking daemon stackinclude an efficient memory management module that keeps track of thememory consumed by the network protocols (e.g., TCP buffering)associated with the daemon. Depending on memory usage, the memorymanagement module indicates to the kernel whether there is active workperformed by the networking protocols on behalf of the application.Specifically, if the memory usage increases a certain threshold set bythe memory management module, then the module indicates to the kernelthat active work is being performed by the network protocols on behalfof the application. This lets the kernel know that the increased memoryusage by the daemon is expected. Once the memory associated with thebuffers has returned back to the memory management module, the moduleindicates to the system that the active work is complete. This preventsthe system from prematurely targeting daemon processes that consume morememory while doing active work.

More generally, while the foregoing process is described within thecontext of a network daemon, artisans of ordinary skill in the relatedarts will readily appreciate, given the contents of the presentdisclosure, that substantially similar techniques could be used on otherapplications (e.g., slow responding or infrequent) and/or otherprotocols with longer queuing intervals.

As previously alluded to, user space networking stacks introduce aunique problem in the operation of daemons. For example, a user stackdaemon (e.g., a daemon running TCP/IP stack) is hidden from the kernelbecause it is an agent located within user space. Additionally, the userstack daemon is also hidden from the user space application because itoften provides functionality that requires special access permissions,etc. Accordingly, user space networking creates the unusual problem thatthe kernel may “kill” a properly functioning user stack daemon if thedaemon appears to be leaking memory (i.e., is corrupted).

For example, in the context of TCP/IP operation, the packetretransmission memory is handled by a user stack daemon. TCP/IP requiresordered packet delivery to the application. If a packet is receivedout-of-order (i.e., packets were lost), the correctly received TCP/IPpackets are held internally until the missing packets that complete theordered set of packets, are retransmitted and received. Once the entiresequence is received (or a portion of the sequence that is otherwise inorder), it can be provided to (or received from) the user spaceapplication. The storing of correctly received but out-of-order TCP/IPpackets may “grow” depending on network conditions. Under certainsituations, this could appear as a memory leak, resulting in the killingof that ongoing TCP/IP process.

FIG. 10A illustrates problematic operation and exemplary operation foruser space communication stacks 1006, while FIG. 10B illustrates anexemplary solution to the problematic operation of user spacecommunication stacks 1006. In FIG. 10A, one exemplary system 1000includes two exemplary user space applications 1002, 1004. Userapplication #1 1002 includes a legacy BSD stack 1010 that operates inkernel space (provided for reference); while daemon #N includes a userspace communications stack 1006. The legacy BSD stack 1010 includes aTCP reassembly queue 1012 that is also present in kernel space; whilethe user space communications stack 1006 includes a TCP reassembly queue1008 that is present within user space. A memory accounting module 1014operating in kernel space is utilized to determine potential memoryleaks within the daemon 1004.

As shown in FIG. 10A, the TCP/IP daemon's reassembly queue 1008 can growin user space, this can look like a runaway process for the memoryaccounting module 1014. The TCP reassembly queue 1008 includes a userspace segment allocator for memory. The segment based allocator tracksmemory requests for e.g., the TCP/IP daemon (associated with the TCPreassembly queue 1008). However, because the TCP/IP daemon operates inuser space, the memory accounting module 1014 operating in kernel spacemay “kill” the daemon should the TCP reassembly queue 1008 exceed athreshold size. In other words, the memory accounting module 1014 maydecide to “prune” the memory allocation for the TCP reassembly queue1008 associated with user space daemon #N 1004. In other words, a suddengrowth in TCP reassembly queue caused by active use in poor networkconditions could be mistaken for a memory leak by the kernel.

For reference, the TCP reassembly queue 1012 associated with the legacyBSD stack 1010 during poor network conditions would expand, and exceed arunaway process threshold.

FIG. 10B illustrates a system 1050 that implements a solution to theproblem introduced by the introduction of user space communicationsstacks 1006. The TCP/IP daemon's reassembly queue 1008 may indicate tothe memory accounting module 1014 that the user space communicationstack 1006 is active (or inactive). In instances where the signaling1016 is indicative of an active user space communications stack 1006,the memory accounting module 1014 may allow the process to continuedespite, for example, the TCP reassembly queue exceeding its memoryallocation.

In another embodiment, the segment based allocator for user space memorymay use virtually identical software (e.g., with application specificmodifications) of the kernel based segment allocator associated withlegacy BSD stacks 1010 to user space TCP/IP. In other words, a secondinstance of the segment allocator (the first instance being located inkernel space) may perform the same memory tracking (e.g.,arena-region-segment-object type memory tracking) for the user spaceTCP/IP process. The second instance may have a “virtual” amount ofmemory that it requests to be backed by physical memory (if necessary).Unlike the traditional kernel based malloc (the generic memory leakkiller de-allocator), the user space segment allocator may be tailoredspecifically for the TCP/IP application considerations. For example, theuser space segment allocator can be set with active/inactive thresholds(via signaling 1016) that are based on TCP/IP protocols, and/or allocateuser space buffers based on TCP/IP packet size (e.g., 1500 byte packets,etc.).

In some implementations, a TCP/IP process may request user space bufferswhen packet retransmission is required. The segment based allocator mayrequest a segment from the kernel, resulting in the generation of anumber of user space buffers. These user space buffers may be providedto the TCP/IP daemon. If the memory allocation associated with theseuser space buffers crosses a threshold value (i.e., where they would bein danger of being pruned), the segment based allocator notifies thekernel that the user process is active via signaling 1016. Thissignaling 1016 indicates to the memory accounting module 1014 that theseuser space buffers should not be pruned unless the TCP reassembly queueappears to be in a runaway active process. The segment based allocatormay also notify the kernel if it is inactive via signaling 1016.Accordingly, the memory accounting module 1014 may establish a differentthreshold value as compared with the threshold value associated withactive processes. In some implementations, the threshold value for aninactive process will be lower than a threshold value for an activeprocess. In other words, the pruning is not disabled, rather differentmetrics may be utilized in order to determine whether the process is“runaway”.

While the prior discussion focused mainly on TCP/IP retransmission, itwould be readily understood by one of ordinary skill given the contentsof the present disclosure that the foregoing discussion may be readilyapplied to any memory allocation/deallocation system that may take intoconsideration active/inactive thresholding.

Other TCP Specific Considerations

TCP presents specific problems for traditional “defunct” Channel I/O(see e.g., co-owned U.S. patent application Ser. No. 16/146,916 filedSep. 28, 2018 and entitled “Methods and Apparatus for Channel DefunctWithin User Space Stack Architectures”, the contents of which werepreviously incorporated herein by reference supra). As a brief aside,during normal TCP operation, TCP packets are received and re-ordered.The TCP packet headers are checked in order to ensure that the receivedTCP packets are ordered correctly.

In other words, since the TCP packets are stored in the channel space,defuncting a channel I/O may result in data inconsistencies (and/orunknown states) in the user space TCP check logic (due to the memorymapping redirection to zero-filled pages); instead of triggering agraceful termination, the user space networking stack could wronglytrigger retransmission attempts or other undesirable data handling.

In one exemplary embodiment, in order to avoid data inconsistency issueswhen a channel is defunct, the user space networking stack copies theoriginal TCP header into stack/heap memory (which is not part of thechannel allocation), before TCP processing begins for the packet. Aftercopying the header, user space TCP checks if the channel is defunct. Ifthe channel did not defunct, then the copied TCP header is deemed valid.Once TCP processing begins, the user networking stack can use the copyof the TCP header to make decisions (thereby preventing undesirablebehavior).

Additionally, in some cases, various embodiments also prevent datacorruption to higher layers above TCP. For example, if the data containszeroes due to memory redirection, then the data is not forwarded on.Instead, after the copy step (from channel buffer to applicationbuffer), user space TCP checks to see if the channel is defunct and ifso indicates that the connection is disconnected (so that data can bethrown away).

More generally, while the foregoing process is described within thecontext of TCP, artisans of ordinary skill in the related arts willreadily appreciate, given the contents of the present disclosure, thatsubstantially similar techniques could be used on any dedicated logic(which would not recognize the aforementioned invalid data).

In other words, using the channel memory after defunct could lead todata inconsistencies in user space TCP. To avoid data inconsistencyissues when a channel is defunct during processing of TCP packet,various embodiments make a shadow copy of the original TCP header instack/heap memory. Once TCP processing begins, it uses the copy of theTCP header to make decisions which prevents any inconsistency or datacorruption. The validation is done prior to handing off the payload datato the layer above TCP, as well as within the TCP input processingpaths.

FIG. 11 is a logical block representation illustrating an exemplaryoperating scenario that has been generalized to any sort of controlplane data (e.g., TCP/IP packet headers). FIG. 11 illustrates anexemplary legacy BSD stack 1010 associated with user space application#1 1002 that operates in conjunction with user space application #N 1004that possesses its own user space communications stack 1102. Whilelegacy BSD is shown being utilized in conjunction with user spacecommunications stacks, it would be readily apparent to one of ordinaryskill given the contents of the present disclosure that the legacy BSDstack 1010 is not required for the operation of the system 1100.

As a brief aside, the terms “foreground” and “background” as usedherein, refer to the priority assigned to programs running in, forexample, a multitasking computing environment. For example, foregroundapplications are applications that a user is currently interacting with(e.g., viewing, providing input to, etc.), while background applicationsare applications that a user is not currently interacting or whereportions of the application have otherwise been suspended. As usedherein, the term “suspended” within the present context refers to anapplication or process in which the given application or process may beremoved from main memory and placed into bulk storage.

Referring back to FIG. 11, when user space application #N 1004 is movedinto the background, after previously being placed into the foreground,the user space communications stack 1102 may be pruned so as to, interalia, free up memory resources. In such a scenario, the control planedata associated with the pruned user space communication stack 1102(e.g., the channel has been defuncted as is described in co-owned U.S.patent application Ser. No. 16/146,916 filed Sep. 28, 2018 and entitled“Methods and Apparatus for Channel Defunct Within User Space StackArchitectures”, the contents of which were previously incorporatedherein by reference supra) and the control plane packet headers arechecked, the control plane payload data is filled with null data 1104(e.g., zero-filled pages) and the control plane packet header is copied1106 into heap/stack memory 1108. As described supra, heap/stack memory1108 is not part of the channel allocation.

Subsequent user space control plane operations will now reference thecopied control plane packet header information 1106 in heap/stack memory1108. When the user space application #N 1004 is resumed (i.e., movedback to the foreground), the user space process resumes operation in thedefuncted, for example, user space TCP/IP stack. The pruned user spacecommunication stack 1102 will see a network connectivity error as aresult of the null data (e.g., zero-filled pages, etc.) and/or a channelschema flag and the pruned user space communication stack 1102 restartsas the connectivity is assumed to be lost and that all of the localinformation will be assumed to be stale. In some implementations, thepruned user space communications stack 1102 does not check the channelschema, and will instead rely on detection of the null data and e.g.,the appropriate TCP/IP header information. These and other variantswould be readily apparent to one of ordinary skill given the contents ofthe present disclosure.

Reaping Based on Heuristics

In-kernel network stacks juggle many different threads simultaneouslyfor the entire system, each having different levels of priority.Historically, it has been unfeasible to identify particular threadswhich are idle or underutilized in a traditional network stack.Consequently, in-kernel resource management (i.e., “inline in datapath”) suffers from lock ordering issues and/or performance loss due toexclusive locks.

In contrast, a user space network stack only services a single thread,thus a user space networking stack can easily identify if its resourcesare being squandered. Moreover, even where the user space networkingstack incorrectly reaps its resources, the resulting performance loss isisolated to itself; it will not affect other stacks or drivers.

In one exemplary embodiment, a user networking stack monitors a numberof parameters and/or other heuristics to determine whether or not theconnection is idle. Common examples of such heuristics may include timealive, time waiting, buffering data, last time active, historic use,predicted use, and/or any other predictive or probabilistic scheme toidentify when to reap a process.

It is appreciated that aggressive reaping methods may be used to improveperformance up to a point; thereafter overly aggressive reaping may bedetrimental. More directly, from a holistic system view, each of theuser space networking stacks is associated with its own unique memorypools per channel and/or per device driver. Each of which has differentperformance requirements. For example, a cellular driver and an Ethernetdriver each have different requirements and/or costs of loss (e.g.,Ethernet typically has higher runtime data rates compared to cellular,and thus a larger memory pool). Consequently, the aggressiveness orconservativeness of process reaping may be fine-tuned based on the typeof application or driver and/or other application specific criteria.

In other words, efficiently and aggressively pruning and purging of idleresources is needed. Various disclosed embodiments include mechanismswhich can detect idle resources and can offload pruning and purging ofthese resources in a deferred context.

Referring to FIG. 12, a driver (or application) 1212 may be allocated aninput-output memory management unit (IOMMU) 1240. In some variants, theIOMMU 1240 or a portion thereof may be required to be wired (i.e.,backed with physical pages). As a brief aside, many drivers mayread/write to memory independently of the operating system (i.e., theO/S cannot swap back the memory). Traditionally speaking, an IOMMU is atype of memory management unit (MMU) that connects a DMA-capable I/O busto physical addresses within memory. However, in the context of thepresent disclosure, the IOMMU 1240 for, for example, a driver (orapplication) 1212 is only capable of accessing a virtual address space(e.g., a driver (or application) arena 1234) within the system MMU 1232,based on a kernel virtual address (KVA) 1230. While a single driverarena 1234 is illustrated in FIG. 12, it would be readily apparent toone of ordinary skill given the contents of the present disclosure thattwo or more driver arenas may be allocated to the driver (orapplication) 1212 (via the IOMMU 1240). In some implementations, fromthe perspective of the driver (or application) 1212, the driver (orapplication) is allocated a contiguous block of memory within the MMU1232.

The KVA 1230 provides the necessary translation between a virtualaddress space within the driver/application arena 1234 and physicaladdresses 1220 in memory. For example, as illustrated in FIG. 12, thedriver/application arena 1234 within the KVA 1230, points to twocontiguous physical address spaces in memory (i.e., physical addressspace A 1222 and physical address space B 1224). Physical address spaceA 1222 may constitute “wired” memory, while physical address B 1224 mayconstitute purgeable memory. Accordingly, if driver/application 1212doesn't require the full amount of memory allocated in itsdriver/application arena 1234, the KVA 1230 may deallocate physicaladdress B 1224. If the driver/application 1212 subsequently requiresadditional memory, the KVA 1230 may reallocate physical address B 1224,or may even allocate to another physical address space (e.g., physicaladdress C 1226). These and other variants would be readily apparent toone of ordinary skill given the contents of the present disclosure.

As shown in FIG. 13, a driver/application arena 1234 is subdivided intoa plurality of regions 1314, each region 1314 is further sub-dividedinto a plurality of segments 1316, and each segment is sub-divided intoa plurality of objects 1318. During an initial channel allocation, thechannel may be allocated objects 1318 to hold packet buffers. However,at any given time, the driver/application may not use all of itsassigned packet buffers, thus these unused packet buffers can be purgedto make more space to other drivers/applications. In legacy BSDoperation, these packet buffers could be purged/reallocated on demandwithout significant penalty. In other words, a single packet buffercould be freed and immediately re-allocated without incurringsignificant additional costs. However, in the context of user spacecommunication stacks as is described within the present disclosure, thefreeing/reallocating of objects 1318, segments 1316, regions 1314 and/orarenas 1234 may result in unnecessary memory churn.

This unnecessary memory churn is resultant from, for example, contextswitching (as described elsewhere herein), between user space and kernelspace. For example, one exemplary example of memory churn occurs whenutilizing a transmission buffer to send a packet to be transmitted. Oncethe packet has been sent, the transmission buffer can then be freed (inuser space) and re-allocated to a receive packet buffer to receive areceive packet by a kernel process. Such a deallocation/reallocation,when performed on demand can be computationally expensive due to, forexample, the aforementioned context switching. Accordingly, in someimplementations, channel allocations (with a corresponding arena 1234,regions 1314, segments 1316 and objects 1318) may be freed or reapedbased on idleness and deferred when possible so as to minimizeunnecessary memory churn.

For example, freeing could occur based on a freed segment 1316, or itsconstituent objects 1318. When and object 1318 or segment 1316 is freed,then the backing memory (e.g., the physical addresses/pages 1220) can befreed in the kernel. However, this freeing of the backup memory 1220 mayoccur on a deferred context basis. For example, the deferred context mayoccur upon the expiration of a timer with the freeing of the backupmemory 1220 occurring on a batch basis. As but yet another example, thedeferred context may occur upon a context switch that has otherwise beeninitiated for a purpose other than for the primary purpose of freeingbackup memory 1220. The use of a deferred context avoids unnecessaryallocation/deallocation memory churn. For example, consider a freedtransmission packet buffer that waits for the deferred context timer toexpire. Once this timer expires, the freed transmission packet buffercan be deallocated in physical memory 1220. If a new use is available tobe allocated (e.g., for use as a receive packet buffer), this freedtransmission packet buffer can be reallocated. These and other variantswould be readily apparent to one of ordinary skill given the contents ofthe present disclosure.

Purgeable Memory (Compressible and Swappable)

The networking memory requirement on, for example, an iOS device can besignificant. Existing architectures needs all of the memory to be wired,which reduces the system's ability to recover under memory pressure asthe memory cannot be swapped or compressed. In a traditional BSD stack,the entire kernel address was “wired”. So-called wired memory is alwayspresent at the same physical (and virtual) memory address, it is neverpurgeable, and the wiring of this memory has traditionally occurred on a“segment” basis. However, by placement of these traditional BSD stacksinto user space and allocating a given user space application its ownuser space communications stack, wiring memory for each of these userspace communication stacks has become extremely memory intensive andexpensive. In fact, dependent upon the number of user spacecommunications stacks implemented, wiring each of these user spacecommunications stacks may even be considered impossible to implement.

Referring back to FIG. 13, and in one exemplary embodiment, the userspace network stack infrastructure architecture allocates all memory as“purgeable” and only wires memory on demand when needed. In someimplementations, a user space networking stack can instantiate multiplestacks in purgeable user space. Drivers may need to DMA (and may requireand IOMMU 1240); consequently, driver allocations may be wired down onan as-needed basis.

So-called purgeable memory is memory that may or may not have a physicalbacking page 1220 in memory. If the physical backing page 1220 can bepurged, the contents of this purged memory may be compressed and movedto, for example, long term storage (e.g., a hard disk drive or othernon-volatile memory storage device). However, the KVA 1230 may bemaintained, even though the virtual address in the KVA 1230 is no longerbacked up by a physical address 1220. When the contents of this purgedmemory are needed again, the compressed memory may be uncompressed andplaced back into a physical memory address 1220. Although the physicalmemory address 1220 may have (and likely would have) changed, thevirtual memory address in the KVA 1230 may remain the same. In otherwords, in some implementations, purgeable data may always be accessiblevia the same virtual address even though the physical page 1220 thatbacks the virtual address may have changed. Notably, DMA always requireswired memory, within the purgeable memory space, because the device mayaccess the memory even when, for example, a processor may notnecessarily be made aware of the access.

Memory Region/Arena: Purpose, Layout, Access Protection, Sharing Model

An efficient and generic mechanism to represent and manage the sharedmemory objects of varying types and sizes which are memory mapped to theuser space and/or kernel space is needed. The traditional BSD stack onlyoffered a single generic interface, also known as a socket. However, byproviding multiple user space communications stacks, more granularityand tweaks to the behavior of the user space communication stack may bebetter optimized for the application that it serves.

In one embodiment, the user space network stack infrastructurearchitecture uses shared memory for efficient packet I/O, networkstatistics and system attributes (sysctl). The user space network stackinfrastructure arena is a generic and efficient mechanism to representthese various types of shared memory subsystems and their backing memorycaches, regions and access protection attributes. Channel schema is arepresentation of the shared memory layout for user space process to beable to efficiently access various channel objects.

A plurality of differing pre-set channel schemas may be established thatare dependent upon a given application's desired mode of operation. Achannel schema may be thought of as a memory sharing agreement betweenthe kernel space and user space, also referred to as a “shared memoryarea”. Referring now to FIG. 14, one exemplary implementation for asystem 1400 for use in accordance with embodiments of the presentdisclosure is now shown and described in detail. The system 1400 maytake on any number of electronic device forms including, withoutlimitation, a desktop computer, a laptop computer, a tablet, a smartphone, an audio/visual computer, smart wearable devices, and othercomputerized devices. For example, the system 1000 may be embodiedwithin any of the Assignee's products (e.g., MacBook®, iMac®, iPad®,Apple Watch®, Apple TV® and iPhone®, etc.).

This exemplary system 1400 may be implemented through the use of anon-transitory computer-readable medium (e.g., a computer-readableapparatus) which may be embodied as software, hardware, or combinationsof the foregoing. The non-transitory computer-readable medium mayinclude one or more computer programs with computer-executableinstructions, that when executed by, for example, one or more processingapparatus may implement one or more of the methodologies describedherein. Moreover, while a specific architecture is shown in FIG. 14, itwould be readily apparent to one of ordinary skill given the contents ofthe present disclosure that the illustrated topology shown in, forexample, FIG. 14 may be readily modified to include one or moreapplications 1004, one or more channels 1404, one or more pool ofresources 1406 associated with a respective application, one or moreflow switches 1408, one or more pool of resources 1410 managed by, forexample, one or more drivers 1412. These and other variants would bereadily understood by one or ordinary skill given the contents of thepresent disclosure with the illustration contained within FIG. 14 merelybeing exemplary.

FIG. 14 illustrates three applications 1004A, 1004B, and 1004C thatreside within user space. One or more of these applications 1004A,1004B, and 1004C may include its own communications stack as isdescribed in additional detail supra. Each of these applications 1004A,1004B, and 1004C may further communicate with the kernel space throughrespective channels 1404A, 1404B, and 1404C which are coupled with arespective pool of dedicated resources 1406A, 1406B, and 1406C. Some (orall) of the data resident within these pools of dedicated resources1406A, 1406B, and 1406C may be communicated to managed pools ofresources 1410A, 1410B via a flow switch apparatus 1408. Each ofapplications 1004A, 1004B, 1004C may operate in accordance with a uniquechannel schema.

In one exemplary embodiment, the flow switch 1408 apparatus isresponsible for transferring data between pools 1406 and pools 1410. Insome implementations, the flow switch apparatus 1408 may read data fromone pool resource (e.g., pool 1406A) and write this data to another poolresource (e.g., pool 1410B) and vice versa. As an alternativeimplementation, the flow switch apparatus may redirect a pointer so asto enable data to be transferred from one pool resource (e.g., pool1410B) to another pool resource (e.g., pool 1406A) and vice versa. Insome variants, this data may be compressed prior to transfer anddecompressed prior to being read and/or may be encrypted prior totransfer and decrypted prior to being read. These and other variantswould be readily apparent to one of ordinary skill given the contents ofthe present disclosure.

Each driver 1412A, 1412B may be assigned (or otherwise possess), its ownrespective IOMMU 1240A, 1240B. Each IOMMU 1240A, 1240B may communicatewith an MMU (1232, FIG. 12) associated with the kernel virtual address1230. Each driver 1412A, 1412B may be assigned an arena 1234, which iscomposed of regions 1314. Each region 1314 may be further sub-divided bysegments (or slabs) 1316. Within each segment (or slab) 1316 may be oneor more objects 1318. While the present disclosure is presented in thecontext of one packet pool scheme, others may be substituted withequivalent success. More directly, any comparable data structure may besubstituted with the arena, region, segment, object being merelyexemplary.

Each packet pool (pool of resources) 1406A, 1406B, 1406C may havedistinct properties from other ones of the packet pools (or pools ofresources). For example, latency and/or throughput may be correlated asa function of segment size. Additionally, the ability to flexibly adjustthe size of the arena 1234, the region 1314, the segment 1316, and theobjects 1318 in order to allocate (or deallocate) memory resources maybe dependent upon the sizing for these respective elements. For example,a segment 1316 may only be freed (e.g., re-allocated), once all theobjects 1318 within the segment 1316 have been freed. In other words, asingle packet located within an object 1318 (and segment 1316) may nothave yet been successfully sent, and accordingly, this single packet mayprevent an entire segment 1316 from being reclaimed. Similarly, allsegments 1316 must be freed for a region 1314 to be freed (e.g.,re-allocated), and all regions 1314 must be freed for an arena 1234 tobe freed (e.g., re-allocated).

A given channel 1404 may request a schema from the nexus. Multiplechannel schemas are possible with each of these multiple channel schemashaving different configurations and policies. Each channel schema may bethought of as a memory sharing agreement between the kernel and userspace. This memory sharing agreement results in the creation of a sharedmemory area. Each shared memory area may be pre-divided into a number ofdata structures that may include a transmission buffer (e.g., a TX ringbuffer), a receive buffer (e.g., a RX ring buffer), an allocation buffer(e.g., an “alloc” ring buffer), and a free buffer (e.g., a free ringbuffer). Each of these buffers is described in co-owned and co-pendingU.S. patent application Ser. No. 16/363,495 filed Mar. 25, 2019 andentitled “Methods and Apparatus for Dynamic Packet Pool Configuration inNetworking Stack Infrastructures”, the contents of which beingincorporated herein by reference in its entirety. Each of these buffersmay be individually configurable dependent upon application 1004requirements. For example, in the context of ring buffers (e.g., TX, RX,‘alloc’, and free), a shorter/smaller ring size may have faster latencyresponse, but may also have smaller amount of data throughput.Conversely, a longer/larger ring size may have slower latency response,but may also have a higher amount of data throughput. Intermediate ringsizes may balance latency and throughput dependent upon the application1004 requirements.

Each schema may include optimized flow advisor settings, as well asdiffering operating statistics. A given schema may have a number ofdifferent size allocations for the arena 1234, individual regions 1314,individual segments 1316 and individual objects 1318. Each of these sizeallocations may be optimized for a given application 1004. Additionally,memory offsets, access rights (e.g., read/write permissions) and othermemory considerations may be selected for a given channel schema. Inaddition to pre-set schemas that may be requested (or given) to a givenapplication 1004, in some implementations, an application 1004 mayrequest a custom setting for the pre-selected channel schema chosen. Thenexus may have different priorities in place in order to determinewhether to grant, deny, or modify a given custom setting request. Forexample, an allocated schema may be different than the channel's requestfor a schema (e.g., as a result of not having enough memory resources).Conversely, an allocated schema may be the same as that requested by thechannel 1404. In some implementations, the allocated schema may includea continuous channel allocation (e.g., a continuous allocation ofvirtual addresses within the kernel virtual address 1230). Such variantsmay have advantages in terms of memory management complexity.Conversely, in some implementations, the allocated schema may include anon-contiguous channel allocation. This non-contiguous channelallocation may allow, for example, a developer of an application 1004 togrow individual regions separately (e.g., the ring buffer sizes may bedynamically changed separate from the rest of the memory allocations inthe channel schema, etc.). Some implementations may implement versioningsupport for various pre-selected channel schemas. Accordingly, adeveloper may develop an application 1004 that is intended to operate inaccordance with a given schema, but may later further develop theapplication 1004 to support a newer version of the given schema. Theseand other variants would be readily apparent to one of ordinary skillgiven the contents of the present disclosure.

Mirrored Memory Regions—

To implement security validation and sanitation of shared memory objectson the user-kernel boundary, kernel checks a kernel only copy of theseobjects. Improved methods for allocating and retrieving these objectsare needed.

In one embodiment, the system creates mirrored memory objects whichshare the same region offset as that of the associated object and hencecan be retrieved quickly from the attributes of the associated object.

As a brief aside, implementation of communication stacks within the userspace as is described elsewhere herein may introduce unique securityvulnerabilities. For example, a so-called “TOCTOU” attack is a class ofsoftware bugs that are caused by differences between the “Time of Check”(TOC) and the “Time of Use” (TOU) of data. Traditionally, a kernelprocess would check user data before use, in order to ensure that thedata was valid (or “clean”). For various reasons, there may be a gap intime between the TOC and the TOU. Accordingly, in a TOCTOU attack, anattacker may repeatedly write a malicious value to the user data.Although the kernel process would catch most of these malicious writes,eventually a malicious write may hit the gap between the TOC and theTOU. This may be particularly problematic where each user stack processcreates the network packets themselves (as is described elsewhereherein). For example, a malicious attacker may use a TOCTOU attack towrite a malformed packet in order to, e.g., access the kernel process orotherwise attack another network entity. More directly, since networkingstacks (e.g., TCP/IP) are in user space in implementations of thepresent disclosure, it is conceivable that packets may be generated thatare not otherwise allowed from that particular TCP/IP instance. In otherwords, user space communication stacks may be susceptible to maliciousactivity and could generate packets that don't belong to a given sourceIP address/source port address, etc. The prevention of IP address/portspoofing is described in co-owned and co-pending U.S. patent applicationSer. No. 16/146,324 filed Sep. 28, 2018 and entitled “Methods andApparatus for Preventing Packet Spoofing with User Space CommunicationStacks”, the contents of which being incorporated herein by reference inits entirety.

Accordingly, security validation of data may be required such that thetime of check and the time of use are not compromised in order toprevent against so-called TOCTOU attacks. This may entail the copying ofdata (e.g., packets) from user space to kernel space and vice versa. Asa brief aside, when the user space application generates a packet, thekernel allocates and creates: a metadata kernel object (MDK), and ametadata user object (MDU). Initially, the MDU (written by the userspace) may be copied into an MDK data structure. More directly, the userspace only has access to the MDU, whereas the kernel space has access toboth the MDU and MDK. The MDU and MDK are “parallel” but distinctobjects; thus, only certain fields within the objects are transferredduring a write from user space to kernel space (internalizing data fromthe MDU to the MDK) and reads from kernel space to user space(externalizing data from the MDK to the MDU). However, the MDK must besanitized before it can be used in the kernel space at transaction.

As used herein, the term “sanitize”, “sanitization”, and/or “sanitizing”refers to a process of ensuring that data conforms to the privilegesand/or requirements of the kernel and/or the user space. For example,when internalizing data from the MDU to the MDK, the data may be checkedfor appropriate formatting, validity, and/or malicious content (e.g., toavoid security and stability issues as discussed supra). Similarly, whenexternalizing data from the MDK to the MDU, the data may be checked toensure that kernel private flags, values, and/or other sensitiveinformation are not unintentionally exposed. If the MDK is successfullysanitized, then an optimized MDK may be generated from the sanitizedMDK. For example, the MDK may be re-written to “naturally align” withthe “natural word boundaries” of the processor cache. This sanitizationprocess is described in further detail within co-owned and co-pendingU.S. patent application Ser. No. 16/236,032 filed Dec. 28, 2018 andentitled “Methods and Apparatus for Classification of Flow Metadata withUser Space Communication Stacks”, the contents of which beingincorporated herein by reference in its entirety.

Efficient memory allocation for MDU and MDK operation is a problem thatis unique to the user space networking configuration of user spacecommunication stacks and described elsewhere herein. Notably, searchingthrough a table to identify empty memory allocations to support a MDUand MDK individually is inefficient. In other words, one would need toscan the entire memory for an arbitrary allocation of the MDU and MDK.Similarly, schemes based on referential data (e.g., if the MDU had apointer to MDK), would necessarily require dereferencing the data fromone to identify a location in the other (which would also beinefficient).

As shown in FIG. 15, a common offset from a base address for both theMDU and the MDK are used between the user-accessible region 1500 and thekernel-only accessible region 1502. As previously discussed above, thekernel may have access to both the user accessible region 1500 andkernel-only accessible region 1502, while user space applications wouldonly have access to the user accessible region 1500. For example, onemay use the same format and same offset (mirrored) between the region1314A and the mirrored region 1314B. The same may also be appliedbetween segment 1316A and the mirrored segment 1316B, and/or the object1318A and the mirrored object 1318B. In other words, if the MDU addressis known (offset+user base address), then the MDK address is the sameoffset (offset+kernel base address). The mirroring may always be synced,which aids in finding a new memory allocation (i.e., either the userspace or the kernel space knows the memory spaces that are available inthe other—each process can infer availability in the other from theirmirrored memory map).

User Pipe Dynamic Memory Management Using Sync Stats

Various embodiments of the user pipe nexus provide an efficient IPCbetween user space processes using shared memory. However, the number ofprocesses using IPC on an iOS device can be significant. An efficientmechanism is needed to keep the shared memory usage to minimum withoutcompromising on the data throughput.

In one embodiment, the system maintains a fair estimate of immediatememory usage of user (working set) depending on the recent past usage.User pipe nexus maintains a weighted moving average statistics of memoryused during each sync and keeps adjusting the channel memoryaccordingly.

Methods—

FIG. 16 is a logical block diagram of a generalized method 1600 forallocating memory for channels and reclaiming memory from inactive/deadchannels.

At step 1602 of the method 1600, a non-kernel space application opens achannel. In some implementations, the non-kernel space application is auser space application, while in other implementations; the non-kernelspace application is a driver application. In one exemplary embodiment,the opening of the channel is associated with one or more applicationspecific considerations. For example, these considerations may include,for example, those considerations described supra (e.g., see Memoryregion/arena: purpose, layout, access protection, sharing model —). Inother words, the opening of the channel results in the non-kernel spaceapplication requesting (or being allocated) a channel schema. Thechannel schema can be selected from one of a plurality of differentpre-designated channel schemas. In some variants, the channel schemachosen may be modified in response to requests made from the non-kernelspace application. For example, the non-kernel space application mayrequest modifications be made to a memory size for one or more of thearena, the region, the segment and/or the objects associated with thenon-kernel space application. More generally, a channel may beestablished by a flow switch apparatus between two or more of devices,user accounts, applications, user space kernel space, driver spacekernel space, user space driver space, other entities, and combinationsof the foregoing.

At step 1604 of the method 1600, the kernel creates a virtual space forthe channel established at step 1602. In one exemplary embodiment, thevirtual space includes resources for communication via a networkprotocol. The network protocol may include, for example, TCP/IP. In oneexemplary embodiment, the resources are a memory space. For example, thememory space may include a kernel virtual address space where the kernelvirtual address space (or portions thereof), may be backed by physicaladdresses (pages) in memory at step 1606. In some implementations, theseresources may also include buffer sizes, buffer types, accesspermissions and/or other types of channel statistics (e.g., latencyand/or throughput). In some implementations, the channel schema mayfurther define various address offsets. For example, one address offsetmay be taken from an absolute address. In some variants, the addressoffset may be taken from a relative base address.

In some implementations, the memory allocations are mirrored betweenuser space and kernel space. See also, for example, Mirrored memoryregions—described supra. In one exemplary embodiment, some portions ofthe virtual space may be “wired”, while other portions may be“purgeable”. As used herein, “wired” memory refers to memory allocationsthat are backed by actual physical memory; in contrast, “purgeable”memory refers to memory allocations that may be either actually presentor virtually present (virtually present memory can be recalled from alarger backing memory, with a cache lookup penalty). In someimplementations, the entirety of the virtual space may constitutepurgeable memory.

At step 1606 of the method 1600, the kernel space allocates one or morephysical memory pages to back at least a portion of the virtual spacewhen the virtual space is in use. In some variants, memory allocationsthat are needed for the currently active user space stack are wired. Inother variants, driver allocations are wired in addition to, oralternatively from, the currently active user space stack. In stillother variants, some set of prioritized applications are wired. Theseprioritized applications may be determined based on prior usage,historical usage, and/or other determined metrics. In someimplementations, use may be explicitly identified based on an “active”signaling. In one such variant, the active signaling may be based on auser space daemon. See also, for example, the discussion with regards toFIG. 10B described supra. In other embodiments, use may be implicitlydetermined based on metrics and/or other monitoring.

At step 1608 of the method 1600, the kernel space determines whether ornot the virtual space is being under-utilized and/or misused. Forexample, this determination may be made and implemented in accordancewith the discussion surrounding FIG. 11 described supra. In someimplementations, some portions of memory cannot be deallocated andhence, the determination made at step 1608 will be ‘no’. For example,some portions of the virtual space may be required to be wired (i.e.,always backed by physical memory).

In some implementations, memory use is tracked and dependent upon memoryusage, the networking stack associated with a given application may bereclaimed (i.e., deallocated at step 1610). Memory usage may beidentified using one or more statistics. In one example, when a userstack is idle, the user stack can be reclaimed at step 1610. In onevariant, reclamation of the user stack may be reclaimed in accordancewith a deferred context as is described supra. In other implementations,memory use is tracked and dependent upon results of runaway processing,and a determination of runaway processing may result in one or morephysical memory pages being deallocated (e.g., reaped) at step 1610.

In some variants, some memory can never be deallocated. For example, inaddition to the aforementioned wired memory, memory that is in activeuse by, for example, a TCP/IP process cannot be deallocated. In thecontext of TCP/IP processing, the deallocation of one or more physicalmemory pages may be restricted to the TCP/IP payload, while the TCP/IPheader information may be stored in heap memory. See also the discussionwith respect to FIG. 11 described supra. The storage of TCP/IP headerinformation along with, for example, zero filled pages in heap memorymay enable the device to see that the connection has been dropped in acontrolled way.

It will be recognized that while certain embodiments of the presentdisclosure are described in terms of a specific sequence of steps of amethod, these descriptions are only illustrative of the broader methodsdescribed herein, and may be modified as required by the particularapplication. Certain steps may be rendered unnecessary or optional undercertain circumstances. Additionally, certain steps or functionality maybe added to the disclosed embodiments, or the order of performance oftwo or more steps permuted. All such variations are considered to beencompassed within the disclosure and claimed herein.

While the above detailed description has shown, described, and pointedout novel features as applied to various embodiments, it will beunderstood that various omissions, substitutions, and changes in theform and details of the device or process illustrated may be made bythose skilled in the art without departing from principles describedherein. The foregoing description is of the best mode presentlycontemplated. This description is in no way meant to be limiting, butrather should be taken as illustrative of the general principlesdescribed herein. The scope of the disclosure should be determined withreference to the claims.

What is claimed is:
 1. A system for managing pools of resources, thesystem comprising: one or more processor apparatus; one or moreapplications executable by the one or more processor apparatus; physicalmemory for use by the one or more applications; and an input/outputmemory management unit (IOMMU) for use by the one or more applications,the IOMMU configured to access a kernel virtual address (KVA), the KVAproviding for translation of a virtual address accessed by the one ormore applications to a physical address associated with the physicalmemory; wherein the one or more processor apparatus are configured to:allocate a virtual address space to one of the one or more applications;associate the virtual address space with a portion of the physicalmemory via use of the IOMMU; and de-allocate a subset of the portion ofthe physical memory associated with the virtual address space.
 2. Thesystem of claim 1, wherein the virtual address space comprises an arena,the arena comprising a plurality of regions, each of the plurality ofregions comprising a plurality of segments, each of the plurality ofsegments comprising a plurality of objects.
 3. The system of claim 2,wherein the deallocated subset of the portion of the physical memory isassociated with a first segment of the plurality of segments.
 4. Thesystem of claim 3, wherein the deallocation occurs in accordance with adeferred context, and the deferred context comprises an expiration of atimer.
 5. The system of claim 3, wherein the deallocation occurs inaccordance with a deferred context, and the deferred context comprises acontext switch between user space and kernel space.
 6. The system ofclaim 1, wherein the deallocation of the subset of the portion of thephysical memory occurs responsive to signaling from a user process to akernel process.
 7. The system of claim 6, wherein the signaling from theuser process to the kernel process is indicative that the one of the oneor more applications is in an active state, the active state associatedwith an active threshold value.
 8. The system of claim 7, wherein thesignaling from the user process to the kernel process is indicative thatthe one of the one or more applications is in an inactive state, theinactive state associated with an inactive threshold value, the inactivethreshold value being lower than the active threshold value.
 9. Thesystem of claim 1, wherein the one or more processor apparatus arefurther configured to: place the one of the one or more applicationsfrom a foreground process to a background process; receive control planepacket data, the received control plane packet data comprised of controlplane header data and control plane payload data; replace the controlplane payload data with null data; and copy the null data and thecontrol plane header data into heap memory.
 10. The system of claim 9,wherein the one or more processor apparatus are further configured to:read the null data and the control plane header data in the heap memory;and resume network connectivity for the one of the one or moreapplications.
 11. A method for requesting a channel schema for anapplication, the method comprising: requesting the channel schema forthe application; and assigning a plurality of data structures associatedwith the requested channel schema, a portion of the plurality of datastructures being configured in accordance with application requirementsfor the application.
 12. The method of claim 11, wherein the assigningof the plurality of data structures comprises: assigning a transmissionbuffer having a first buffer size; and assigning a receive buffer havinga second buffer size.
 13. The method of claim 12, further comprisingreceiving a request for a different size for one or both of the firstbuffer size and the second buffer size.
 14. The method of claim 12,wherein the receiving of the request for the different size for one orboth of the first buffer size and the second buffer size comprisesincreasing a data throughput for one or both of transmission buffer andthe receive buffer.
 15. The method of claim 12, wherein the receiving ofthe request for the different size for one or both of the first buffersize and the second buffer size comprises decreasing an amount oflatency for one or both of transmission buffer and the receive buffer.16. The method of claim 11, further comprising: mirroring the pluralityof data structures between a user-accessible region and a kernel-onlyaccessible region; and applying a common offset from a base addresswithin the user-accessible region and the kernel-only accessible regionfor a metadata user object and a metadata kernel object associated withthe metadata user object.
 17. A method for deallocating one or morephysical memory pages from a kernel virtual address space, comprising:assigning the kernel virtual address space to an application; backingthe kernel virtual address space with physical memory; determining thata portion of the kernel virtual address space is underutilized; anddeallocating the portion of the kernel virtual address space that isunderutilized from the physical memory.
 18. The method of claim 17,wherein the deallocating of the portion of the kernel virtual addressspace that is underutilized from the physical memory comprisescompressing contents from the physical memory that is associated withthe portion of the kernel virtual address space that is underutilized;and placing the compressed contents from the physical memory into anon-volatile memory storage device.
 19. The method of claim 18, furthercomprising accessing the portion of the virtual address space that hasbeen deallocated, the accessing comprising decompressing the compressedcontents in the non-volatile memory storage device; and placing thedecompressed contents in the non-volatile memory storage device into thephysical memory.
 20. The method of claim 19, wherein the placing of thedecompressed contents in the non-volatile memory storage device into thephysical memory comprises placing the decompressed contents into anotherarea of the physical memory, while the accessing of the portion of thevirtual address space comprises using a same virtual address spaceaddressing for the contents from the physical memory.